-- Logs begin at Thu 2022-06-16 16:29:56 UTC, end at Thu 2022-06-16 17:42:22 UTC. -- Jun 16 17:08:05.314380 dv-cinder-837356-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 17:08:05.325255 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 17:08:05 2022] *** Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: nodename: dv-cinder-837356-2-iscsi Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: machine: x86_64 Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: clock source: unix Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: pcre jit disabled Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: detected number of CPU cores: 4 Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: current working directory: / Jun 16 17:08:05.326623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: detected binary path: /usr/bin/uwsgi-core Jun 16 17:08:05.327200 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: your processes number limit is 47680 Jun 16 17:08:05.327200 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: your memory page size is 4096 bytes Jun 16 17:08:05.327200 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: detected max file descriptor number: 2048 Jun 16 17:08:05.327200 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: lock engine: pthread robust mutexes Jun 16 17:08:05.327200 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: thunder lock: enabled Jun 16 17:08:05.327200 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 17:08:05.327200 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 17:08:05.372676 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: Python main interpreter initialized at 0x55d21e44f9c0 Jun 16 17:08:05.372676 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: python threads support enabled Jun 16 17:08:05.372676 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: your server socket listen backlog is limited to 100 connections Jun 16 17:08:05.372676 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: your mercy for graceful operations on workers is 90 seconds Jun 16 17:08:05.373023 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 17:08:05.373098 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: *** Operational MODE: preforking *** Jun 16 17:08:05.373393 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: *** uWSGI is running in multiple interpreter mode *** Jun 16 17:08:05.373393 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: spawned uWSGI master process (pid: 80230) Jun 16 17:08:05.373601 dv-cinder-837356-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 17:08:05.374623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: spawned uWSGI worker 1 (pid: 80238, cores: 1) Jun 16 17:08:05.374623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: spawned uWSGI worker 2 (pid: 80239, cores: 1) Jun 16 17:08:05.374623 dv-cinder-837356-2-iscsi devstack@keystone.service[80230]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 17:08:10.816107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:08:10.816107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes Jun 16 17:08:10.816931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:08:10.816931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes Jun 16 17:08:10.850048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:08:10.850048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: from cryptography.utils import int_from_bytes Jun 16 17:08:10.850671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 17:08:10.850671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: from cryptography.utils import int_from_bytes Jun 16 17:08:10.867856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 17:08:10.868024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 17:08:10.868209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] command line args: [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 17:08:10.868532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 17:08:10.868672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ================================================================================ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 17:08:10.868962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.869212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_dir = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.869437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.869593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] config_source = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.869804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] debug = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.869994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.870379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.870532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.870802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.871054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.871284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.871457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_config_append = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.871628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.871807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_dir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.871987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.872226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.872475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.872724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.872972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.873153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.873357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.873606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.873855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.874091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.874326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.874560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.874796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.875043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.875293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.875571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.876067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.876251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] publish_errors = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.876482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.876716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.876896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.877142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.877408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.877643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] standard_threads = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.877878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.878121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.878301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.878478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_journal = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.878655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_json = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.878831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_stderr = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.879009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] use_syslog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.879239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.879497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.879737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.880004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.880245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.880496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.880797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.881035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.881303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.881556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.881805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.882061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.882297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.882545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.882785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.883033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.883283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.883522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.883780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.884035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.884283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.884522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.884776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.885040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.885320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.885573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.885809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.886043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.886276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.886513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.886762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.887012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.887245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.887477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.887730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.887980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.888219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.888471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.888705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.888944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.889195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.889436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.889669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.889906 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.890138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.890377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.890629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.890863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.891101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.891336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.891572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.891809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.892057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.892288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.892526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.892764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.893001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.893263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.893505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.893739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.893989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.894227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.894464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.894699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.894960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.895199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.895434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.895675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.895914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.896185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.896462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.896810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.897343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.897601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.897846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.898088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.898168 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 17:08:10.898341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.898422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 17:08:10.898576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] command line args: [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 17:08:10.898704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.898876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.898952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 17:08:10.899023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.899092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ================================================================================ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 17:08:10.899294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.899510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] admin_token = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.899644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.899729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.899895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] config_dir = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.900016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.900183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.900304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.900438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.900526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] config_source = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.900702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.900833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] debug = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.900917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.901000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.901197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.901394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.901515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.901667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.901787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.901888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.902052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.902173 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.902347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.902466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.902544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.902735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.902825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.903004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.903085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_config_append = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.903234 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.903404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.903493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.903703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.903783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_dir = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.903952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.904033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_file = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.904192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.904377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.904458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.904641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.904721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.904896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.905066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.905141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.external = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.905254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.905328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.905409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.905532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.905678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.905886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.905968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.906119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.password = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.906345 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.906422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.token = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.906585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.906695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.906823 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.906942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.907072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.907184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.907305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.907425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.907555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.907678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.907802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.907929 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.908036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.908209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.908315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.908508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.908715 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.66/identity {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.908841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.908942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] publish_errors = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.909042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.909167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.909308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.909440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.909517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.909630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.909705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.909823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.909956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.910035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.910205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] standard_threads = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.910284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.910438 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.910553 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.910683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.910809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.910886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.911223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.911299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_journal = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.911299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_json = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.911425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.911546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_stderr = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.911625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.911704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] use_syslog = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.911780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.911851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 17:08:10.911964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.912101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.912271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.912392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.912508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.912587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.912752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.912838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.913068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.913147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.913334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.913413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.913586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.913684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.913826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.913905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.914077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.914153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.914313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.914398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.914569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.914655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.914807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.914885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.915055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.915140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.915292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.915371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.915537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.915615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.915785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.915870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.916977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.917053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.917250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.917346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.917509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.917593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.917746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.917824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.917979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.918984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.919156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.919233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.919399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.919475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.919627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.919704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.919860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.919944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.920110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.920188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.920350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.920427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.920585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.920661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.920817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.920894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.921054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.921139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.921310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.921388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.921548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.921624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.921785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.921894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922569 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.922964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.923964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.924119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.924197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.924355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.924440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.924584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.924669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.924821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.924901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.925059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.925137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.925316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.925395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.925554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.925634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.925789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.925867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.926966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.927045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.927197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.927276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.927429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.927515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.927693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.927797 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.927939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.928042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.928172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.928283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.928419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.928500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.928665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.928742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.928926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.929944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.930100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.930179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.930333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.930412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.930568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.930646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.930810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.930888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.931974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.932992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.933153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.933288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.933415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.933496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.933652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.933729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.933886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.934492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.934570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.934641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.934711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.934781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.934852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.934930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935903 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.935990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.936152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.936230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.936405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.936482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.936642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.936727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.936875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.external = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.936953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.937108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.937210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.937398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.937492 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.937639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.937718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.937870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.password = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.937947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.938105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] auth.token = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.938191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.938344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.938421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.938577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.938661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.938822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.938907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.939051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.939128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.939297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.939373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.939540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.939624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.939773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.939851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.940975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.941065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.941232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.941313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.941473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.941550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.941720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.941810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.941968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.942047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.942201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.942278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.942452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.942623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.942737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.942862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.942983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.943113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.943261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.943407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.943528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.943658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.943900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.943977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.944948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.945103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.945196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.945375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.945470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.945610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.945689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.945862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.945940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.946114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.946200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.946353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.946431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.946588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.946665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.946825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.946910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.947055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.947133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.947292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.947370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.947530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.947617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.947763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.947843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.948968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.949058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.949223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.949305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.949451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.949529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.949682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.949772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.949925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.950959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.951126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.951212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.951376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.951452 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.951612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.951690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.951843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.951931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.952079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.952154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.952295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.952374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.952529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.952607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.952771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.952850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.953997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.954076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.954230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.954307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.954470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.954555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.954713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.954790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.954943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.955958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 17:08:10.956132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.956342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.956580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.956827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.957073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.957332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.957564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.957814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.958062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.958298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.958531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.958764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.959040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.959280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.959541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.959774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.960023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.960595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.960843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.961078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.961340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.961578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.961815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.962055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.962288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.962537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.962783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.963030 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.963278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.963524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.963754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.964004 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.964243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.964480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.964718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.964955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.965232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.965476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.965716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.965974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.966207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.966454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.966693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.966930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.967167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.967416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.967659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.967892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.968131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.968373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.968623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.968873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.969112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.969389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.969644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.969882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.970130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.970366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.970605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.970841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] role.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.971094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] role.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.971326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.971576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.971827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.972075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.972324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.972570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.972816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.973063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.973342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.973581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.973829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.974079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.974326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.974684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.975133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.975518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.975774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.976026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.976276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.976528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.976765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.977021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.977306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.977551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.977784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.978019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.978252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.978503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.978761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.979011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.979249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.979485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.979717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.979954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.980191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.980443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.980681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.980938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.981223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.981453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.981701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.981945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.982193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.982429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.982671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.982906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.983153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.983402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.983632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.983875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.984198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.984462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.984707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.984958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.985214 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.985460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.985696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.985935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.986173 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.986411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.986656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.986894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.987142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.987381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.987632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.987869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.988116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.988349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.988589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.988825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 17:08:10.988998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80239) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 17:08:11.073513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.073727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.073831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.082528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.100056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.100265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.100487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.109027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.111203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.111416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.127216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.127468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.164696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.166360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.213202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.213414 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.214580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.278855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.279987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.280121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.280242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.303111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.303246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.303530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.303628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.303777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.303859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.303979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.304071 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.304192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.304325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.304448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.304527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.304646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.304739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.315412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.315539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.328951 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.329107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.329278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.329379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.329447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80239) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.329530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 17:08:11.346934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.347287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:08:11.347853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.348092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:08:11.348366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.348625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.348707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.348881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.348964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:08:11.349134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.349249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.349435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 17:08:11.349816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.350196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.350302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.350562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.360060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.360344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.360614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.360888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.361410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.361691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.361955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.362231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.369384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.370807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.372660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.372941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.373204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.373522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.373801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.374060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.374181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.374325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.374475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.374643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.374778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.374902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.375026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.375155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.375275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.375401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.375534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.375670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.375903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.376169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.376440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.376709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.402489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.403560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.404087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.404371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.405216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.405501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.409662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.409944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.410344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.410845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.411249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.411532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.417734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.418018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.419084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.419365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.422521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.422809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.423084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.423359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.423628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.423822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.423948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.424134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.424257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.424403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.424539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.424684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.424931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.425228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.425523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.425807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.443125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.443446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.444443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.444718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.448003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.449355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.451686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.451978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.452948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.453373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.457055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.458312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.461117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.462410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.464789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.465092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.465403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.465682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.465957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.466079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.466241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.466369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.466656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.466931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.467219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.467492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.487976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.487976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.487976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.487976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.488316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.488316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.488316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.488316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.494584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.496285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.496451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.496550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.498035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.498311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.501442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.501729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.502009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.502290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.502575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.502947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.503241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.503528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.503811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.504099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.515946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.516230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.516511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.516786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.517164 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.517404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.517527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.517665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.517745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.517973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.518252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.518628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.518905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.519180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.536071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.537124 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.539525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.540458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.542863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.543780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.544595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.545523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.546279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.546563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.546845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.547164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.547451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.547744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.554210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.555034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.555824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.556642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.559074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.559359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.559749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.559870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.560006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.560107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.560528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.560805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.569236 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.569531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.569818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.570010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.570321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.570600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.577706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80239) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.577985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.578617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.578733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 17:08:11.578869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.578989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.579067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.579293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.579418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.579552 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.579724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.579847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.579971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.580190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.580314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.580447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.580598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.580762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.580900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.581021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.581112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.581337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.581460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.581580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80239) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.581671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.581872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.582138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.582405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 17:08:11.619665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d21e44f9c0 pid: 80239 (default app) Jun 16 17:08:11.620438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d21e44f9c0 pid: 80238 (default app) Jun 16 17:08:11.623377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e680-69c5-480e-a172-0ab1948c3640 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:11.623668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e680-69c5-480e-a172-0ab1948c3640 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:11.623947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69e680-69c5-480e-a172-0ab1948c3640 None None] PATH_INFO: `/v3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:11.628360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1/1] 10.222.0.66 () {50 vars in 791 bytes} [Thu Jun 16 17:08:11 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:19.632098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1adb-d7b6-4c55-ae71-ecfdfa25f8cf None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:19.633060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1adb-d7b6-4c55-ae71-ecfdfa25f8cf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:19.633060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe1adb-d7b6-4c55-ae71-ecfdfa25f8cf None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:19.640288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1/2] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:19 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:19.649797 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:19.650228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:19.650373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:19.731629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: INFO dbcounter [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] Registered counter for database keystone Jun 16 17:08:19.757079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG oslo_db.sqlalchemy.engines [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80239) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 17:08:19.764084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writer thread running {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 17:08:19.949304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG passlib.handlers.bcrypt [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80239) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 17:08:19.949735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG passlib.handlers.bcrypt [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80239) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 17:08:20.006512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:20.008608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:20.142563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 2/3] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 17:08:20.303791 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe1adb-d7b6-4c55-ae71-ecfdfa25f8cf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:20.376843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: INFO dbcounter [None req-02fe1adb-d7b6-4c55-ae71-ecfdfa25f8cf None None] Registered counter for database keystone Jun 16 17:08:20.398318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG oslo_db.sqlalchemy.engines [None req-02fe1adb-d7b6-4c55-ae71-ecfdfa25f8cf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80238) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 17:08:20.409666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writer thread running {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 17:08:20.538493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe1adb-d7b6-4c55-ae71-ecfdfa25f8cf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:20.543480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5482c1a4-d0ef-49b5-af01-ca5b2fbf2396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:20.544150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c1a4-d0ef-49b5-af01-ca5b2fbf2396 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:20.544435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c1a4-d0ef-49b5-af01-ca5b2fbf2396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:20.544717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482c1a4-d0ef-49b5-af01-ca5b2fbf2396 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:20.559573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482c1a4-d0ef-49b5-af01-ca5b2fbf2396 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:20.617632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5482c1a4-d0ef-49b5-af01-ca5b2fbf2396 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 17:08:20.617954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5482c1a4-d0ef-49b5-af01-ca5b2fbf2396 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:20.628737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 2/4] 10.222.0.66 () {58 vars in 1324 bytes} [Thu Jun 16 17:08:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:08:20.638947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:20.664252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e715d20-421a-4bb0-87d9-8b8878b64fda None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:20.670034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38810e-4658-441c-a068-243e87750da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:20.671165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38810e-4658-441c-a068-243e87750da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:20.671456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38810e-4658-441c-a068-243e87750da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:20.671736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38810e-4658-441c-a068-243e87750da7 admin admin] PATH_INFO: `/v3/projects/546ad1026cd44cbea571239c12f41a3f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:20.673392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38810e-4658-441c-a068-243e87750da7 admin admin] RBAC: Authorizing `identity:get_project(project_id=546ad1026cd44cbea571239c12f41a3f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:20.723401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8a38810e-4658-441c-a068-243e87750da7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 17:08:20.723742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38810e-4658-441c-a068-243e87750da7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:20.728731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 3/5] 10.222.0.66 () {56 vars in 1207 bytes} [Thu Jun 16 17:08:20 2022] GET /identity/v3/projects/546ad1026cd44cbea571239c12f41a3f => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:22.810754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e43333-2e62-45d2-b6eb-cd991169f8f4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:22.811579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e43333-2e62-45d2-b6eb-cd991169f8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:22.811579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e43333-2e62-45d2-b6eb-cd991169f8f4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:22.818346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 3/6] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:22.823873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4948e6c-5eb9-4e0d-ad73-c56e3b108560 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:22.824289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4948e6c-5eb9-4e0d-ad73-c56e3b108560 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:22.824389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4948e6c-5eb9-4e0d-ad73-c56e3b108560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:22.844529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f4948e6c-5eb9-4e0d-ad73-c56e3b108560 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:22.846090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f4948e6c-5eb9-4e0d-ad73-c56e3b108560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:22.855016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 4/7] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 17:08:22.888250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e43333-2e62-45d2-b6eb-cd991169f8f4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:22.901630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e43333-2e62-45d2-b6eb-cd991169f8f4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:22.905982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92620e3c-0fca-4cc3-85d0-1422a880c3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:22.906632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92620e3c-0fca-4cc3-85d0-1422a880c3ab admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:22.906920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92620e3c-0fca-4cc3-85d0-1422a880c3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:22.907201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92620e3c-0fca-4cc3-85d0-1422a880c3ab admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:22.916295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92620e3c-0fca-4cc3-85d0-1422a880c3ab admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:22.918209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92620e3c-0fca-4cc3-85d0-1422a880c3ab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:22.926997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 4/8] 10.222.0.66 () {58 vars in 1324 bytes} [Thu Jun 16 17:08:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:08:22.933060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4948e6c-5eb9-4e0d-ad73-c56e3b108560 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:22.945876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4948e6c-5eb9-4e0d-ad73-c56e3b108560 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:22.951833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa9cf07-f11a-4524-9b08-0414bd1dc244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:22.952815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9cf07-f11a-4524-9b08-0414bd1dc244 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:22.953098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9cf07-f11a-4524-9b08-0414bd1dc244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:22.953405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9cf07-f11a-4524-9b08-0414bd1dc244 admin admin] PATH_INFO: `/v3/users/918e7fd9b64348e38ccb56232b8e098a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:22.955014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9cf07-f11a-4524-9b08-0414bd1dc244 admin admin] RBAC: Authorizing `identity:get_user(user_id=918e7fd9b64348e38ccb56232b8e098a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:22.956904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa9cf07-f11a-4524-9b08-0414bd1dc244 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:22.958598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 5/9] 10.222.0.66 () {56 vars in 1198 bytes} [Thu Jun 16 17:08:22 2022] GET /identity/v3/users/918e7fd9b64348e38ccb56232b8e098a => generated 247 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:08:25.144785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb8e2-d79e-4b45-9bcf-c09d588b11a3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.145364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb8e2-d79e-4b45-9bcf-c09d588b11a3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.145364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb8e2-d79e-4b45-9bcf-c09d588b11a3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.146849 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 5/10] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:25.152436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617a6e-a453-4458-b970-3af4c3498632 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.152865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617a6e-a453-4458-b970-3af4c3498632 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.152945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73617a6e-a453-4458-b970-3af4c3498632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.171177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-73617a6e-a453-4458-b970-3af4c3498632 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:25.172497 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-73617a6e-a453-4458-b970-3af4c3498632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:25.174209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2a5ed-bf3b-4f17-8ef6-cc23fb3e9003 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.174695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2a5ed-bf3b-4f17-8ef6-cc23fb3e9003 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.174976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2a5ed-bf3b-4f17-8ef6-cc23fb3e9003 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.177152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 6/11] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:08:25.182681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.182981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.183261 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.224202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80238) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 17:08:25.224631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80238) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 17:08:25.258210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 6/12] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:25.270881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:25.272682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:25.281388 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73617a6e-a453-4458-b970-3af4c3498632 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:25.306641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:25.375351 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 7/13] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:25.391890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73617a6e-a453-4458-b970-3af4c3498632 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:25.396916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:25.427298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a5125-1dcd-4158-aa68-89e6ef4ffce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:25.428099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a5125-1dcd-4158-aa68-89e6ef4ffce9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.428367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a5125-1dcd-4158-aa68-89e6ef4ffce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.428732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a5125-1dcd-4158-aa68-89e6ef4ffce9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.439704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5a5125-1dcd-4158-aa68-89e6ef4ffce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5e5a5125-1dcd-4158-aa68-89e6ef4ffce9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:25.450899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:25.451874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:25.452717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 7/14] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:08:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:08:25.458248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a5125-1dcd-4158-aa68-89e6ef4ffce9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:25.502921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae33707-ccf9-47ff-9af5-9fd89b9084c0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:25.506189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:25.542557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ab006-304c-48d0-be5e-863d7afcf429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:25.543463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab006-304c-48d0-be5e-863d7afcf429 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.543752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab006-304c-48d0-be5e-863d7afcf429 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.544036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ab006-304c-48d0-be5e-863d7afcf429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.547570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:25.566072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5a5125-1dcd-4158-aa68-89e6ef4ffce9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:25.600077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:25.600840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.601124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.601435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.602198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:25.602510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:25.603917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:25.612212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ab006-304c-48d0-be5e-863d7afcf429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:25.613858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ab006-304c-48d0-be5e-863d7afcf429 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:25.615999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 8/15] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:08:25.621953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:25.717025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 8/16] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:08:25.722743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ab006-304c-48d0-be5e-863d7afcf429 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:25.726009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:25.728018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eb36d-a63d-4670-b7be-5ab99c6ad5fb None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:25.766794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfecac-9109-4db5-bd0c-e144edfbea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:25.767704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfecac-9109-4db5-bd0c-e144edfbea90 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.767987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfecac-9109-4db5-bd0c-e144edfbea90 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.768263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfecac-9109-4db5-bd0c-e144edfbea90 None admin] PATH_INFO: `/v3/users/918e7fd9b64348e38ccb56232b8e098a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.769566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfecac-9109-4db5-bd0c-e144edfbea90 None admin] RBAC: Authorizing `identity:get_user(user_id=918e7fd9b64348e38ccb56232b8e098a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:25.771034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfecac-9109-4db5-bd0c-e144edfbea90 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:25.773065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 9/17] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:25 2022] GET /identity/v3/users/918e7fd9b64348e38ccb56232b8e098a => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:25.778248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfecac-9109-4db5-bd0c-e144edfbea90 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:25.793882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:25.841602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ab006-304c-48d0-be5e-863d7afcf429 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:25.880583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:25.881545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.881926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.882205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.884376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfecac-9109-4db5-bd0c-e144edfbea90 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:25.901012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:25.902566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:25.917757 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d6f5e-8084-452c-a96c-dba27bc27beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:25.918554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6f5e-8084-452c-a96c-dba27bc27beb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:25.918840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6f5e-8084-452c-a96c-dba27bc27beb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:25.919117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d6f5e-8084-452c-a96c-dba27bc27beb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:25.920334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d6f5e-8084-452c-a96c-dba27bc27beb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:25.921870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d6f5e-8084-452c-a96c-dba27bc27beb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:08:25.923281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:25.924241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:25.925704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:08:25.925704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:08:25.925704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:08:25.925704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:25.926158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 10/18] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:08:25.926245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 9/19] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:25.930551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d6f5e-8084-452c-a96c-dba27bc27beb None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:25.930832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:25.988799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:26.014674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:26.033883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d6f5e-8084-452c-a96c-dba27bc27beb None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:26.034964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a7da4-76c2-4cb7-8468-c973c03b22a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:26.068440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17dc3b-836a-4103-bd11-a03e71b77539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:26.069131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17dc3b-836a-4103-bd11-a03e71b77539 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:26.069468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17dc3b-836a-4103-bd11-a03e71b77539 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:26.069747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17dc3b-836a-4103-bd11-a03e71b77539 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:26.070246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509d156d-6c5b-4fd5-aa75-5eec44233c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:26.070472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17dc3b-836a-4103-bd11-a03e71b77539 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:26.070777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17dc3b-836a-4103-bd11-a03e71b77539 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:26.070895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d156d-6c5b-4fd5-aa75-5eec44233c9b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:26.071187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d156d-6c5b-4fd5-aa75-5eec44233c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:26.071464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509d156d-6c5b-4fd5-aa75-5eec44233c9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:26.072161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17dc3b-836a-4103-bd11-a03e71b77539 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:26.072286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d156d-6c5b-4fd5-aa75-5eec44233c9b None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.domain.id=default, user.id=918e7fd9b64348e38ccb56232b8e098a) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:26.072611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d156d-6c5b-4fd5-aa75-5eec44233c9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:26.074087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509d156d-6c5b-4fd5-aa75-5eec44233c9b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:26.082833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 10/20] 10.222.0.66 () {56 vars in 1322 bytes} [Thu Jun 16 17:08:25 2022] GET /identity/v3/role_assignments?user.id=918e7fd9b64348e38ccb56232b8e098a&scope.domain.id=default&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:08:26.089861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 11/21] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:28.410416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9bd0-391e-4bf5-a5c8-0733c46825af None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:28.411431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9bd0-391e-4bf5-a5c8-0733c46825af None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:28.412008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d9bd0-391e-4bf5-a5c8-0733c46825af None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:28.413936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 11/22] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:28.419404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e85a1-1aaf-4c64-8317-3ba5a21cb246 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:28.419828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e85a1-1aaf-4c64-8317-3ba5a21cb246 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:28.419996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e85a1-1aaf-4c64-8317-3ba5a21cb246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:28.442564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-bf2e85a1-1aaf-4c64-8317-3ba5a21cb246 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:28.443798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bf2e85a1-1aaf-4c64-8317-3ba5a21cb246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:28.550026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 12/23] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:28.589084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d9bd0-391e-4bf5-a5c8-0733c46825af None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:28.753367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d9bd0-391e-4bf5-a5c8-0733c46825af None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:28.779656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:28.801373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc681c-8fc6-4f2e-ab78-0679ed82fd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:28.802239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc681c-8fc6-4f2e-ab78-0679ed82fd71 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:28.802526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc681c-8fc6-4f2e-ab78-0679ed82fd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:28.802798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc681c-8fc6-4f2e-ab78-0679ed82fd71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:28.816947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac359c3-bec3-4923-96d0-1443a37c8e03 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:28.817587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac359c3-bec3-4923-96d0-1443a37c8e03 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:28.818070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac359c3-bec3-4923-96d0-1443a37c8e03 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:28.819934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 13/24] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:28.823944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fc681c-8fc6-4f2e-ab78-0679ed82fd71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-08fc681c-8fc6-4f2e-ab78-0679ed82fd71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:28.833991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:28.834940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:28.836488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 12/25] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:08:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:28.843454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac359c3-bec3-4923-96d0-1443a37c8e03 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:28.847000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486ffa5-0d19-4b0f-8a5f-fd3eaf9881b0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:28.847334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486ffa5-0d19-4b0f-8a5f-fd3eaf9881b0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:28.847619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486ffa5-0d19-4b0f-8a5f-fd3eaf9881b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:28.877061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b486ffa5-0d19-4b0f-8a5f-fd3eaf9881b0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:28.878460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b486ffa5-0d19-4b0f-8a5f-fd3eaf9881b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:28.970869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:29.009703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac359c3-bec3-4923-96d0-1443a37c8e03 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:29.036010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 13/26] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:08:29.046812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ef9c3-d4de-4e6b-94b3-42a6845c62e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:29.047811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef9c3-d4de-4e6b-94b3-42a6845c62e1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.048312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef9c3-d4de-4e6b-94b3-42a6845c62e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.048780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ef9c3-d4de-4e6b-94b3-42a6845c62e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.049810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ef9c3-d4de-4e6b-94b3-42a6845c62e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:29.050305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ef9c3-d4de-4e6b-94b3-42a6845c62e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:29.051910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20ef9c3-d4de-4e6b-94b3-42a6845c62e1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:29.061661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 14/27] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:29.070789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486ffa5-0d19-4b0f-8a5f-fd3eaf9881b0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:29.076640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65452b04-98c1-4b1e-a612-ceaaec59b58e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.077208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65452b04-98c1-4b1e-a612-ceaaec59b58e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.077679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65452b04-98c1-4b1e-a612-ceaaec59b58e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.079310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 15/28] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:08:29.086927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d2fdc-6668-47a0-9854-43345322e8f7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.087482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d2fdc-6668-47a0-9854-43345322e8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.087948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d2fdc-6668-47a0-9854-43345322e8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.114364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0e7d2fdc-6668-47a0-9854-43345322e8f7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:29.115628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0e7d2fdc-6668-47a0-9854-43345322e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:29.223643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 16/29] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:08:29.224882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d2fdc-6668-47a0-9854-43345322e8f7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:29.230573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486ffa5-0d19-4b0f-8a5f-fd3eaf9881b0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:29.251592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:29.290694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1573fe-7921-48b3-b93d-96f753e8ca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:29.291896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1573fe-7921-48b3-b93d-96f753e8ca9c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.292357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1573fe-7921-48b3-b93d-96f753e8ca9c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.292811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1573fe-7921-48b3-b93d-96f753e8ca9c None admin] PATH_INFO: `/v3/users/918e7fd9b64348e38ccb56232b8e098a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.294628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1573fe-7921-48b3-b93d-96f753e8ca9c None admin] RBAC: Authorizing `identity:get_user(user_id=918e7fd9b64348e38ccb56232b8e098a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:29.296432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1573fe-7921-48b3-b93d-96f753e8ca9c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:29.296821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:29.298614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 14/30] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:29 2022] GET /identity/v3/users/918e7fd9b64348e38ccb56232b8e098a => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:29.299810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1573fe-7921-48b3-b93d-96f753e8ca9c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:29.340506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d2fdc-6668-47a0-9854-43345322e8f7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:29.376854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c351bc-a61a-4b1a-98c4-a44a46ad7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:29.377824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c351bc-a61a-4b1a-98c4-a44a46ad7fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.378105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c351bc-a61a-4b1a-98c4-a44a46ad7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.378377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c351bc-a61a-4b1a-98c4-a44a46ad7fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.389671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c351bc-a61a-4b1a-98c4-a44a46ad7fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:29.391303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c351bc-a61a-4b1a-98c4-a44a46ad7fb0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-a8c351bc-a61a-4b1a-98c4-a44a46ad7fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:08:29.411791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:29.412732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:29.413874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:08:29.413874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:08:29.413874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:08:29.413874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:29.413874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 17/31] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:29.414600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26be4-c882-4b18-9b77-e31261c82801 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.414872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26be4-c882-4b18-9b77-e31261c82801 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.415148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c26be4-c882-4b18-9b77-e31261c82801 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.416572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 18/32] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:29.418042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c26be4-c882-4b18-9b77-e31261c82801 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:29.427466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1573fe-7921-48b3-b93d-96f753e8ca9c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:29.477609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ccdb9-7c35-44ef-8e76-242054f2a699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:29.478524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ccdb9-7c35-44ef-8e76-242054f2a699 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.478940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ccdb9-7c35-44ef-8e76-242054f2a699 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.479344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ccdb9-7c35-44ef-8e76-242054f2a699 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.481317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ccdb9-7c35-44ef-8e76-242054f2a699 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:29.483412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ccdb9-7c35-44ef-8e76-242054f2a699 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:29.557774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c26be4-c882-4b18-9b77-e31261c82801 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:29.573789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:29.597343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a78584-41e1-43b5-9a71-496aa6852683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:29.598514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a78584-41e1-43b5-9a71-496aa6852683 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.598951 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a78584-41e1-43b5-9a71-496aa6852683 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.599377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a78584-41e1-43b5-9a71-496aa6852683 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.600966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a78584-41e1-43b5-9a71-496aa6852683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:29.602673 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a78584-41e1-43b5-9a71-496aa6852683 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:29.604521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 19/33] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:29.605462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a78584-41e1-43b5-9a71-496aa6852683 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:29.647934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 15/34] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Jun 16 17:08:29 2022] POST /identity/v3/roles => generated 225 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:29.650044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f7cf0-f4fa-4938-89f4-7e9601c6df6c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.650490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f7cf0-f4fa-4938-89f4-7e9601c6df6c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.650940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f7cf0-f4fa-4938-89f4-7e9601c6df6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.670830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-280f7cf0-f4fa-4938-89f4-7e9601c6df6c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:29.672703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-280f7cf0-f4fa-4938-89f4-7e9601c6df6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:29.707161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:29.728834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a78584-41e1-43b5-9a71-496aa6852683 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:29.753821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 16/35] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:29.755156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f7cf0-f4fa-4938-89f4-7e9601c6df6c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:29.769629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:29.770493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.770784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.771072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.772012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:29.772491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:29.774493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:29.790598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 20/36] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:29.793137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:29.853594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:29.921809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68936c83-3d60-49ad-946b-ad29ca9f5bad None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:29.942239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f7cf0-f4fa-4938-89f4-7e9601c6df6c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:29.962694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f44f64-2f40-47f5-ab61-8f585f2064d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:29.963539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f44f64-2f40-47f5-ab61-8f585f2064d3 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.965258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f44f64-2f40-47f5-ab61-8f585f2064d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.965794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f44f64-2f40-47f5-ab61-8f585f2064d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.966830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f44f64-2f40-47f5-ab61-8f585f2064d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:29.968537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f44f64-2f40-47f5-ab61-8f585f2064d3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:29.989229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808518-397a-4004-93d6-cca1d0b3fab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:29.990575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808518-397a-4004-93d6-cca1d0b3fab7 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:29.991055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808518-397a-4004-93d6-cca1d0b3fab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:29.991486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22808518-397a-4004-93d6-cca1d0b3fab7 None admin] PATH_INFO: `/v3/domains/default/users/918e7fd9b64348e38ccb56232b8e098a/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:29.994170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22808518-397a-4004-93d6-cca1d0b3fab7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=918e7fd9b64348e38ccb56232b8e098a, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:29.996042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22808518-397a-4004-93d6-cca1d0b3fab7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:30.026755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 17/37] 10.222.0.66 () {58 vars in 1361 bytes} [Thu Jun 16 17:08:29 2022] PUT /identity/v3/domains/default/users/918e7fd9b64348e38ccb56232b8e098a/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:08:30.028019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808518-397a-4004-93d6-cca1d0b3fab7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:30.070697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:30.176913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22808518-397a-4004-93d6-cca1d0b3fab7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:30.221834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032d32-6a0c-4603-aa6a-cbf07047e671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:30.222854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032d32-6a0c-4603-aa6a-cbf07047e671 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:30.223285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032d32-6a0c-4603-aa6a-cbf07047e671 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:30.223709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41032d32-6a0c-4603-aa6a-cbf07047e671 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:30.224648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41032d32-6a0c-4603-aa6a-cbf07047e671 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:30.226615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41032d32-6a0c-4603-aa6a-cbf07047e671 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:30.236230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 21/38] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Jun 16 17:08:29 2022] POST /identity/v3/roles => generated 219 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:08:30.238104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046fea-680d-442e-b079-1b966a613ca1 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:30.238493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046fea-680d-442e-b079-1b966a613ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:30.238876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb046fea-680d-442e-b079-1b966a613ca1 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:30.240695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 22/39] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:30.246763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2d72-d916-458b-9072-d4c088079319 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:30.247182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2d72-d916-458b-9072-d4c088079319 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:30.247562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d2d72-d916-458b-9072-d4c088079319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:30.266975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 18/40] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:08:30 2022] POST /identity/v3/projects => generated 301 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:30.268807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-200d2d72-d916-458b-9072-d4c088079319 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:30.277296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-200d2d72-d916-458b-9072-d4c088079319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:30.359295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 23/41] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:30.385475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032d32-6a0c-4603-aa6a-cbf07047e671 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:30.608915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:30.632352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41032d32-6a0c-4603-aa6a-cbf07047e671 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:30.667522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1a8bb-ba4f-4510-a1e1-eddf322bbeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:30.668223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a8bb-ba4f-4510-a1e1-eddf322bbeb4 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:30.668514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a8bb-ba4f-4510-a1e1-eddf322bbeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:30.668795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a8bb-ba4f-4510-a1e1-eddf322bbeb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:30.669628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1a8bb-ba4f-4510-a1e1-eddf322bbeb4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:30.671046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc1a8bb-ba4f-4510-a1e1-eddf322bbeb4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:30.685690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 19/42] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Jun 16 17:08:30 2022] POST /identity/v3/roles => generated 223 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:30.814449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247011df-9f22-47e8-a50e-f8832b180e8f None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:30.814968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247011df-9f22-47e8-a50e-f8832b180e8f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:30.814968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247011df-9f22-47e8-a50e-f8832b180e8f None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:30.816544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 24/43] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:30.822606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac237c-47e3-4036-af85-b832ddb3194d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:30.822946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac237c-47e3-4036-af85-b832ddb3194d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:30.823157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac237c-47e3-4036-af85-b832ddb3194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:30.841722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-83ac237c-47e3-4036-af85-b832ddb3194d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:30.842975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-83ac237c-47e3-4036-af85-b832ddb3194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:30.917213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 20/44] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:30.940011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247011df-9f22-47e8-a50e-f8832b180e8f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:30.972451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:31.054385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247011df-9f22-47e8-a50e-f8832b180e8f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:31.089230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662004d8-af0a-4315-993f-da8690c3dcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:31.089941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662004d8-af0a-4315-993f-da8690c3dcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:31.090219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662004d8-af0a-4315-993f-da8690c3dcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:31.090493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662004d8-af0a-4315-993f-da8690c3dcc0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:31.091716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662004d8-af0a-4315-993f-da8690c3dcc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:31.093330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662004d8-af0a-4315-993f-da8690c3dcc0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:31.095131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 25/45] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:31.100897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac237c-47e3-4036-af85-b832ddb3194d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:31.169652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:31.202484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac237c-47e3-4036-af85-b832ddb3194d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:31.237544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcef836-3261-4746-a734-dc10a72d4a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:31.238292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcef836-3261-4746-a734-dc10a72d4a3f None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:31.238576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcef836-3261-4746-a734-dc10a72d4a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:31.238963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcef836-3261-4746-a734-dc10a72d4a3f None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:31.239779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcef836-3261-4746-a734-dc10a72d4a3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:31.241774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcef836-3261-4746-a734-dc10a72d4a3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:31.269430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 21/46] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:08:31 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:32.444749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a024d1b-1f46-4c6a-8bbb-584c3ee5e93c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:32.445329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a024d1b-1f46-4c6a-8bbb-584c3ee5e93c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:32.445411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a024d1b-1f46-4c6a-8bbb-584c3ee5e93c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:32.446844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 26/47] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:32.452365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46480c00-c89e-48c3-9f54-70731b825669 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:32.452804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46480c00-c89e-48c3-9f54-70731b825669 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:32.452958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46480c00-c89e-48c3-9f54-70731b825669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:32.471367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-46480c00-c89e-48c3-9f54-70731b825669 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:32.472590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-46480c00-c89e-48c3-9f54-70731b825669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:32.544005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 22/48] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:32.566915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a024d1b-1f46-4c6a-8bbb-584c3ee5e93c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:32.666487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:32.670789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a024d1b-1f46-4c6a-8bbb-584c3ee5e93c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:32.707169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c25324-6cfb-430f-b499-c9fd614a7255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:32.707895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c25324-6cfb-430f-b499-c9fd614a7255 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:32.708177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c25324-6cfb-430f-b499-c9fd614a7255 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:32.708432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c25324-6cfb-430f-b499-c9fd614a7255 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:32.718953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c25324-6cfb-430f-b499-c9fd614a7255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-20c25324-6cfb-430f-b499-c9fd614a7255 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:32.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:32.727503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:32.728274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 27/49] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:08:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:32.733899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46480c00-c89e-48c3-9f54-70731b825669 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:32.787946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:32.836559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46480c00-c89e-48c3-9f54-70731b825669 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:32.871639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:32.872317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:32.872601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:32.872876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:32.873650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:32.873962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:32.875367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:32.887466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 23/50] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:32.893069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c25324-6cfb-430f-b499-c9fd614a7255 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:32.997031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c25324-6cfb-430f-b499-c9fd614a7255 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:33.032422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d347af-75f5-4c9f-b1d0-86d165c12489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:33.033338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d347af-75f5-4c9f-b1d0-86d165c12489 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:33.033623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d347af-75f5-4c9f-b1d0-86d165c12489 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:33.033904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d347af-75f5-4c9f-b1d0-86d165c12489 None admin] PATH_INFO: `/v3/users/918e7fd9b64348e38ccb56232b8e098a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:33.035152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d347af-75f5-4c9f-b1d0-86d165c12489 None admin] RBAC: Authorizing `identity:get_user(user_id=918e7fd9b64348e38ccb56232b8e098a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:33.036676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d347af-75f5-4c9f-b1d0-86d165c12489 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:33.038387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 28/51] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:32 2022] GET /identity/v3/users/918e7fd9b64348e38ccb56232b8e098a => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:33.044002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:33.148881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a5b19-d2c9-4339-a74c-e5dc3d9f8fc2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:33.163293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:33.188217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd262d-8517-45ec-a250-09d4e916fdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:33.189040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd262d-8517-45ec-a250-09d4e916fdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:33.189361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd262d-8517-45ec-a250-09d4e916fdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:33.189638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd262d-8517-45ec-a250-09d4e916fdd1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:33.190854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd262d-8517-45ec-a250-09d4e916fdd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:33.192345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd262d-8517-45ec-a250-09d4e916fdd1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:33.199610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 24/52] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:33.200876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d347af-75f5-4c9f-b1d0-86d165c12489 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:33.230867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:33.302830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d347af-75f5-4c9f-b1d0-86d165c12489 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:33.341239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4728ceb-de36-4098-b390-e188c8eaa529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:33.341926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4728ceb-de36-4098-b390-e188c8eaa529 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:33.342211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4728ceb-de36-4098-b390-e188c8eaa529 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:33.342491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4728ceb-de36-4098-b390-e188c8eaa529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:33.343342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4728ceb-de36-4098-b390-e188c8eaa529 None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.domain.id=default, user.id=918e7fd9b64348e38ccb56232b8e098a) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:33.343659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4728ceb-de36-4098-b390-e188c8eaa529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:33.345133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4728ceb-de36-4098-b390-e188c8eaa529 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:33.352842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 29/53] 10.222.0.66 () {56 vars in 1322 bytes} [Thu Jun 16 17:08:33 2022] GET /identity/v3/role_assignments?user.id=918e7fd9b64348e38ccb56232b8e098a&scope.domain.id=default&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:33.375694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a9c27-60d5-4234-8d60-13713aa81932 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:33.375989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a9c27-60d5-4234-8d60-13713aa81932 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:33.376270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a9c27-60d5-4234-8d60-13713aa81932 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:33.377740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 25/54] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:33.384107 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d152-034a-47e4-8578-b1fd6eb0ab74 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:33.384401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d152-034a-47e4-8578-b1fd6eb0ab74 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:33.384680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1d152-034a-47e4-8578-b1fd6eb0ab74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:33.403551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2ee1d152-034a-47e4-8578-b1fd6eb0ab74 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:33.404832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2ee1d152-034a-47e4-8578-b1fd6eb0ab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:33.481776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 30/55] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:33.506383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a9c27-60d5-4234-8d60-13713aa81932 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:33.621527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a9c27-60d5-4234-8d60-13713aa81932 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:33.656561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70c400-0b72-4879-927e-3bfc174e80a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:33.657410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70c400-0b72-4879-927e-3bfc174e80a8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:33.657698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70c400-0b72-4879-927e-3bfc174e80a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:33.657982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70c400-0b72-4879-927e-3bfc174e80a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:33.659320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70c400-0b72-4879-927e-3bfc174e80a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:33.660804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70c400-0b72-4879-927e-3bfc174e80a8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:33.662532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 26/56] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:33.669233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1d152-034a-47e4-8578-b1fd6eb0ab74 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:33.696276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:33.774732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1d152-034a-47e4-8578-b1fd6eb0ab74 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:33.813196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6b222-2821-4473-b46d-5569496af524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:33.813883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b222-2821-4473-b46d-5569496af524 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:33.814167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b222-2821-4473-b46d-5569496af524 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:33.814447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6b222-2821-4473-b46d-5569496af524 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:33.815156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6b222-2821-4473-b46d-5569496af524 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:33.816629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6b222-2821-4473-b46d-5569496af524 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:33.850607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 31/57] 10.222.0.66 () {60 vars in 1140 bytes} [Thu Jun 16 17:08:33 2022] POST /identity/v3/projects => generated 298 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:35.950217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6246a-54e5-49fb-8319-3fb990f50f9f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:35.951034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6246a-54e5-49fb-8319-3fb990f50f9f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:35.951170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6246a-54e5-49fb-8319-3fb990f50f9f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:35.953461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 27/58] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:35.959869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a8ea-79a4-4ce4-bddc-06783fd88d5a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:35.960594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a8ea-79a4-4ce4-bddc-06783fd88d5a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:35.960862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a8ea-79a4-4ce4-bddc-06783fd88d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:35.989516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b2f5a8ea-79a4-4ce4-bddc-06783fd88d5a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:35.991284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b2f5a8ea-79a4-4ce4-bddc-06783fd88d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:36.063393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 32/59] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:36.093451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6246a-54e5-49fb-8319-3fb990f50f9f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:36.149028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:36.204692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6246a-54e5-49fb-8319-3fb990f50f9f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:36.244423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb53855-a483-41e7-8f96-d27c11c30806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:36.245222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb53855-a483-41e7-8f96-d27c11c30806 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:36.245538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb53855-a483-41e7-8f96-d27c11c30806 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:36.245817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb53855-a483-41e7-8f96-d27c11c30806 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:36.247099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb53855-a483-41e7-8f96-d27c11c30806 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:36.248640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb53855-a483-41e7-8f96-d27c11c30806 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:36.250359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 28/60] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:36.256475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5a8ea-79a4-4ce4-bddc-06783fd88d5a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:36.260113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:36.360668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5a8ea-79a4-4ce4-bddc-06783fd88d5a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:36.398753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543daffd-12de-4bd0-87d3-a84c9fb6278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:36.399418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543daffd-12de-4bd0-87d3-a84c9fb6278c None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:36.399706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543daffd-12de-4bd0-87d3-a84c9fb6278c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:36.399988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543daffd-12de-4bd0-87d3-a84c9fb6278c None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:36.400700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543daffd-12de-4bd0-87d3-a84c9fb6278c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:36.402188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543daffd-12de-4bd0-87d3-a84c9fb6278c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:36.430513 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 33/61] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:08:36 2022] POST /identity/v3/users => generated 275 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:39.726609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0e41e-9297-4d78-8a5e-3987d4f2f7d5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:39.727534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0e41e-9297-4d78-8a5e-3987d4f2f7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:39.727961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0e41e-9297-4d78-8a5e-3987d4f2f7d5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:39.729532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 29/62] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:39.735223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce9a2f-c64c-4f41-9ee9-1843c8b2a53b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:39.736062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce9a2f-c64c-4f41-9ee9-1843c8b2a53b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:39.736488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce9a2f-c64c-4f41-9ee9-1843c8b2a53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:39.762408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7cce9a2f-c64c-4f41-9ee9-1843c8b2a53b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:39.763901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7cce9a2f-c64c-4f41-9ee9-1843c8b2a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:39.843122 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:39.845955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502db0f-693e-435b-8e22-97cf7be311bc None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:39.846294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502db0f-693e-435b-8e22-97cf7be311bc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:39.846596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502db0f-693e-435b-8e22-97cf7be311bc None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:39.847974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 30/63] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:08:39.862401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a706ff5-f029-447b-8f7c-21b0904b613b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:39.862691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a706ff5-f029-447b-8f7c-21b0904b613b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:39.862977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a706ff5-f029-447b-8f7c-21b0904b613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:39.864638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 34/64] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:39.890383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2a706ff5-f029-447b-8f7c-21b0904b613b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:39.890568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce9a2f-c64c-4f41-9ee9-1843c8b2a53b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:39.891638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2a706ff5-f029-447b-8f7c-21b0904b613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:39.991092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 31/65] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 17:08:39.992753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad61cc3-f568-400e-92d6-0be6a3318b0d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:39.993302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad61cc3-f568-400e-92d6-0be6a3318b0d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:39.993716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad61cc3-f568-400e-92d6-0be6a3318b0d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:39.995355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 32/66] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:40.000992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faac7d-9568-4ad1-8ac5-3b5e928714d2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.001319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faac7d-9568-4ad1-8ac5-3b5e928714d2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.001601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5faac7d-9568-4ad1-8ac5-3b5e928714d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.026174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b5faac7d-9568-4ad1-8ac5-3b5e928714d2 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:40.027674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b5faac7d-9568-4ad1-8ac5-3b5e928714d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:40.032531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce9a2f-c64c-4f41-9ee9-1843c8b2a53b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.069052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0806d9-03ac-4360-a8c8-9377a3ba8af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.070173 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0806d9-03ac-4360-a8c8-9377a3ba8af1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.070600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0806d9-03ac-4360-a8c8-9377a3ba8af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.071044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0806d9-03ac-4360-a8c8-9377a3ba8af1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.076512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:40.085511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0806d9-03ac-4360-a8c8-9377a3ba8af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1b0806d9-03ac-4360-a8c8-9377a3ba8af1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:40.093811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:40.094828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:40.095728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:40.097808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 35/67] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:08:40.098708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0806d9-03ac-4360-a8c8-9377a3ba8af1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.102228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 33/68] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:40.110240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faac7d-9568-4ad1-8ac5-3b5e928714d2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.152066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:40.209091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0806d9-03ac-4360-a8c8-9377a3ba8af1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.244625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667a080-e885-4ac9-bebf-58f0f63df7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.245804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667a080-e885-4ac9-bebf-58f0f63df7cc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.246085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667a080-e885-4ac9-bebf-58f0f63df7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.246355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9667a080-e885-4ac9-bebf-58f0f63df7cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.247593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5faac7d-9568-4ad1-8ac5-3b5e928714d2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.258237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9667a080-e885-4ac9-bebf-58f0f63df7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9667a080-e885-4ac9-bebf-58f0f63df7cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:40.267568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:40.268429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 36/69] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:40.274782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667a080-e885-4ac9-bebf-58f0f63df7cc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.289165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c187c158-538e-4cd8-9ef5-1b325c2dcb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.290024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c158-538e-4cd8-9ef5-1b325c2dcb30 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.290380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c158-538e-4cd8-9ef5-1b325c2dcb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.290705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c187c158-538e-4cd8-9ef5-1b325c2dcb30 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.291475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187c158-538e-4cd8-9ef5-1b325c2dcb30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:40.291789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187c158-538e-4cd8-9ef5-1b325c2dcb30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:40.293353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c187c158-538e-4cd8-9ef5-1b325c2dcb30 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:40.306904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 34/70] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:40.306904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6982-0574-414d-ad3b-370ed9df328b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.306904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6982-0574-414d-ad3b-370ed9df328b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.306904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd6982-0574-414d-ad3b-370ed9df328b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.306904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 35/71] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:40.307297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd6982-0574-414d-ad3b-370ed9df328b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.356554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:40.391353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9667a080-e885-4ac9-bebf-58f0f63df7cc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.401926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:40.424406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd6982-0574-414d-ad3b-370ed9df328b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.433462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97938f-204f-4817-b5fd-f435379b80e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.434311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97938f-204f-4817-b5fd-f435379b80e7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.434656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97938f-204f-4817-b5fd-f435379b80e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.434941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97938f-204f-4817-b5fd-f435379b80e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.447454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97938f-204f-4817-b5fd-f435379b80e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8a97938f-204f-4817-b5fd-f435379b80e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:40.455592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:40.456481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:40.457283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 37/72] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:40.457916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97938f-204f-4817-b5fd-f435379b80e7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.468136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382ed07-383d-4908-96b5-d613ba26684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.468833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ed07-383d-4908-96b5-d613ba26684c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.469126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ed07-383d-4908-96b5-d613ba26684c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.469444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382ed07-383d-4908-96b5-d613ba26684c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.470213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ed07-383d-4908-96b5-d613ba26684c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:40.470533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ed07-383d-4908-96b5-d613ba26684c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:40.471917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382ed07-383d-4908-96b5-d613ba26684c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:40.485750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 36/73] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:40.485750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000e94-4158-40c9-8435-26533ec5bb9a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.485750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000e94-4158-40c9-8435-26533ec5bb9a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.485750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7000e94-4158-40c9-8435-26533ec5bb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.500059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c7000e94-4158-40c9-8435-26533ec5bb9a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:40.501309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7000e94-4158-40c9-8435-26533ec5bb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:40.560714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97938f-204f-4817-b5fd-f435379b80e7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.566462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 37/74] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:40.567455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7000e94-4158-40c9-8435-26533ec5bb9a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.594140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a857483-4c5c-424a-b250-e3b993d77cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.595025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a857483-4c5c-424a-b250-e3b993d77cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.595314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a857483-4c5c-424a-b250-e3b993d77cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.595595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a857483-4c5c-424a-b250-e3b993d77cf3 None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.609220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a857483-4c5c-424a-b250-e3b993d77cf3 None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:40.610695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a857483-4c5c-424a-b250-e3b993d77cf3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:40.612392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 38/75] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:40.613774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03f33-b93c-4167-a6e8-14062d9489e7 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.614065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03f33-b93c-4167-a6e8-14062d9489e7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.614347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b03f33-b93c-4167-a6e8-14062d9489e7 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.615739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 39/76] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:40.616325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b03f33-b93c-4167-a6e8-14062d9489e7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.617373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:40.645258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:40.687717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7000e94-4158-40c9-8435-26533ec5bb9a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.721706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b03f33-b93c-4167-a6e8-14062d9489e7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.723271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.723976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.724264 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.724552 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.725355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:40.725670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:40.727244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:40.735923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 38/77] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:40.737688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.765974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850717fe-7589-414f-9333-b337d13aee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.767141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850717fe-7589-414f-9333-b337d13aee16 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.767440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850717fe-7589-414f-9333-b337d13aee16 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.767717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850717fe-7589-414f-9333-b337d13aee16 None admin] PATH_INFO: `/v3/users/918e7fd9b64348e38ccb56232b8e098a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.769710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850717fe-7589-414f-9333-b337d13aee16 None admin] RBAC: Authorizing `identity:get_user(user_id=918e7fd9b64348e38ccb56232b8e098a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:40.772285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850717fe-7589-414f-9333-b337d13aee16 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:40.774691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 40/78] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/users/918e7fd9b64348e38ccb56232b8e098a => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:40.775558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850717fe-7589-414f-9333-b337d13aee16 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.832747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:40.840011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0723a-2d9f-496f-b862-67cc8ac6012b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.871916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:40.873930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d42acd6-bd53-419b-9574-efa65501ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.874706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42acd6-bd53-419b-9574-efa65501ec32 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.874994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42acd6-bd53-419b-9574-efa65501ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.875272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d42acd6-bd53-419b-9574-efa65501ec32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.878262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850717fe-7589-414f-9333-b337d13aee16 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:40.885701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d42acd6-bd53-419b-9574-efa65501ec32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2d42acd6-bd53-419b-9574-efa65501ec32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:40.893320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:40.894335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:40.895275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 39/79] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:40.896064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516a13-f474-4a40-9f5a-79075186ec9e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.896064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516a13-f474-4a40-9f5a-79075186ec9e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.896319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516a13-f474-4a40-9f5a-79075186ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.912890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7b516a13-f474-4a40-9f5a-79075186ec9e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:40.914256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7b516a13-f474-4a40-9f5a-79075186ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:40.915764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26f011-72cb-4ed2-b2d3-a61d8b20e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:40.916420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f011-72cb-4ed2-b2d3-a61d8b20e002 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:40.916699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f011-72cb-4ed2-b2d3-a61d8b20e002 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:40.916975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26f011-72cb-4ed2-b2d3-a61d8b20e002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:40.987581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26f011-72cb-4ed2-b2d3-a61d8b20e002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:40.988094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 40/80] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:40.988941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516a13-f474-4a40-9f5a-79075186ec9e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:40.989393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e26f011-72cb-4ed2-b2d3-a61d8b20e002 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.076997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.088263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.095460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 41/81] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:41.100590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26f011-72cb-4ed2-b2d3-a61d8b20e002 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.100683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516a13-f474-4a40-9f5a-79075186ec9e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.138185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9789-e610-450d-9c88-564482b63b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.139091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9789-e610-450d-9c88-564482b63b8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.139371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9789-e610-450d-9c88-564482b63b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.139820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9789-e610-450d-9c88-564482b63b8b None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.141399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d9789-e610-450d-9c88-564482b63b8b None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.142858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d9789-e610-450d-9c88-564482b63b8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.144493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 41/82] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:40 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:41.145332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9789-e610-450d-9c88-564482b63b8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.203100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e26f011-72cb-4ed2-b2d3-a61d8b20e002 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.238952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f7414-b826-45e9-a196-1f2134981b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.239608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f7414-b826-45e9-a196-1f2134981b83 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.239894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f7414-b826-45e9-a196-1f2134981b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.240172 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f7414-b826-45e9-a196-1f2134981b83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.249975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9789-e610-450d-9c88-564482b63b8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.284749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.285477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.285757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.286033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.286788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:41.287087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.288536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.296904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 42/83] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:41.297642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.303915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.311195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f7414-b826-45e9-a196-1f2134981b83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.312659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f7414-b826-45e9-a196-1f2134981b83 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.313738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.401277 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0ad35-0064-4366-b03d-f02c9e0d7009 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.417369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 42/84] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:41.418223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f7414-b826-45e9-a196-1f2134981b83 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.440265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6c9c9-57f6-48af-bec0-c291ccdfe5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.441070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6c9c9-57f6-48af-bec0-c291ccdfe5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.441379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6c9c9-57f6-48af-bec0-c291ccdfe5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.441657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b6c9c9-57f6-48af-bec0-c291ccdfe5f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.442840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6c9c9-57f6-48af-bec0-c291ccdfe5f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.444342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b6c9c9-57f6-48af-bec0-c291ccdfe5f5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.446078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 43/85] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:41.446842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6c9c9-57f6-48af-bec0-c291ccdfe5f5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.531894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.540013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.540152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f7414-b826-45e9-a196-1f2134981b83 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.559054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b6c9c9-57f6-48af-bec0-c291ccdfe5f5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.575936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f94d0f-7316-48c8-b95e-ccaeda84304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.576817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94d0f-7316-48c8-b95e-ccaeda84304f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.577102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94d0f-7316-48c8-b95e-ccaeda84304f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.577413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f94d0f-7316-48c8-b95e-ccaeda84304f None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.578631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f94d0f-7316-48c8-b95e-ccaeda84304f None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.580132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f94d0f-7316-48c8-b95e-ccaeda84304f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.581894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 43/86] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:41.582684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f94d0f-7316-48c8-b95e-ccaeda84304f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.597020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e478e87-f5aa-43dd-ba90-7dbc4cffac11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.597722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e478e87-f5aa-43dd-ba90-7dbc4cffac11 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.598013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e478e87-f5aa-43dd-ba90-7dbc4cffac11 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.598308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e478e87-f5aa-43dd-ba90-7dbc4cffac11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.667107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e478e87-f5aa-43dd-ba90-7dbc4cffac11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.668513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e478e87-f5aa-43dd-ba90-7dbc4cffac11 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.680931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f94d0f-7316-48c8-b95e-ccaeda84304f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.715520 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ac3b2-0678-48ca-9e37-5311c32e3acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.716393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ac3b2-0678-48ca-9e37-5311c32e3acd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.716674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ac3b2-0678-48ca-9e37-5311c32e3acd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.716950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0ac3b2-0678-48ca-9e37-5311c32e3acd None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.718225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ac3b2-0678-48ca-9e37-5311c32e3acd None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.719786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0ac3b2-0678-48ca-9e37-5311c32e3acd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.721481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 44/87] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:41.722226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ac3b2-0678-48ca-9e37-5311c32e3acd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.753796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.758124 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.779369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 44/88] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:41.780426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e478e87-f5aa-43dd-ba90-7dbc4cffac11 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.831570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0ac3b2-0678-48ca-9e37-5311c32e3acd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.867113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e46d99-eca9-4a49-a285-5059b945bd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.867936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e46d99-eca9-4a49-a285-5059b945bd14 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.868221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e46d99-eca9-4a49-a285-5059b945bd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.868502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e46d99-eca9-4a49-a285-5059b945bd14 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.869760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e46d99-eca9-4a49-a285-5059b945bd14 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.871208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e46d99-eca9-4a49-a285-5059b945bd14 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.873128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 45/89] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:41.873824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e46d99-eca9-4a49-a285-5059b945bd14 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.880997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e478e87-f5aa-43dd-ba90-7dbc4cffac11 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:41.916111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654cee-d61e-4d7a-955f-8acadb45515b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:41.916786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654cee-d61e-4d7a-955f-8acadb45515b None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:41.917068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654cee-d61e-4d7a-955f-8acadb45515b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:41.917446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654cee-d61e-4d7a-955f-8acadb45515b None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:41.918100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83654cee-d61e-4d7a-955f-8acadb45515b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:41.919571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83654cee-d61e-4d7a-955f-8acadb45515b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:41.946473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 45/90] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:08:41 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:41.947490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654cee-d61e-4d7a-955f-8acadb45515b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:41.987674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:41.990721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e46d99-eca9-4a49-a285-5059b945bd14 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:42.002846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:42.040677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:42.041456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:42.041749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:42.042059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:42.042934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] RBAC: Adding query filter params (role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167, user.id=0a40ced9f6d3493d817d4b9d965ce69b) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:42.043253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:42.044745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:42.051101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 46/91] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/role_assignments?user.id=0a40ced9f6d3493d817d4b9d965ce69b&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:42.051961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:42.097048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654cee-d61e-4d7a-955f-8acadb45515b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:42.146431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d1def-3b2f-42e7-bb26-35e649e2b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:42.147135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d1def-3b2f-42e7-bb26-35e649e2b46d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:42.147425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d1def-3b2f-42e7-bb26-35e649e2b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:42.147712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d1def-3b2f-42e7-bb26-35e649e2b46d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:42.214357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0017e5bd-6541-4272-8123-84688f4d5b4f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:42.253707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d1def-3b2f-42e7-bb26-35e649e2b46d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:42.255200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d1def-3b2f-42e7-bb26-35e649e2b46d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:42.261791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3518f-d9f4-4c11-8c64-0011b31583fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:42.262675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3518f-d9f4-4c11-8c64-0011b31583fc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:42.262964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3518f-d9f4-4c11-8c64-0011b31583fc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:42.263242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3518f-d9f4-4c11-8c64-0011b31583fc None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:42.264652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3518f-d9f4-4c11-8c64-0011b31583fc None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:42.266201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3518f-d9f4-4c11-8c64-0011b31583fc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:42.268103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 47/92] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:42 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:42.268750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3518f-d9f4-4c11-8c64-0011b31583fc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:42.304522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:42.313357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:42.387385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 46/93] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:42.388569 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d1def-3b2f-42e7-bb26-35e649e2b46d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:42.406035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3518f-d9f4-4c11-8c64-0011b31583fc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:42.444866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:42.446099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:42.446579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:42.447041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:42.448104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167, user.id=918e7fd9b64348e38ccb56232b8e098a) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:42.448627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:42.450296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:42.457630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 48/94] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:42 2022] GET /identity/v3/role_assignments?user.id=918e7fd9b64348e38ccb56232b8e098a&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:42.458753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:42.518781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d1def-3b2f-42e7-bb26-35e649e2b46d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:42.563708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:42.564396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:42.564685 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:42.564967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:42.569211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] RBAC: Adding query filter params (role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167, user.id=0a40ced9f6d3493d817d4b9d965ce69b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:42.569211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:42.569211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:42.575556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 47/95] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:42 2022] GET /identity/v3/role_assignments?user.id=0a40ced9f6d3493d817d4b9d965ce69b&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:42.626600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:42.637789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a325c-3db5-4e78-a8a4-8a558845d06e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:42.693453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b37f53-70e0-46a3-b700-28324886e19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:42.694839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b37f53-70e0-46a3-b700-28324886e19f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:42.695383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b37f53-70e0-46a3-b700-28324886e19f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:42.695877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b37f53-70e0-46a3-b700-28324886e19f None admin] PATH_INFO: `/v3/projects/7c35c69c19fd4f4cbc44dd22b88228ac` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:42.702126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b37f53-70e0-46a3-b700-28324886e19f None admin] RBAC: Authorizing `identity:get_project(project_id=7c35c69c19fd4f4cbc44dd22b88228ac)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:42.703988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b37f53-70e0-46a3-b700-28324886e19f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:42.706098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 49/96] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:42 2022] GET /identity/v3/projects/7c35c69c19fd4f4cbc44dd22b88228ac => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:42.712933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:42.720073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:42.840049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75647-69fd-47d1-99b2-a6b1d9b3f159 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:42.877275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f193d7-d9a9-4f4c-b24f-5b8ce739af7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:42.877950 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f193d7-d9a9-4f4c-b24f-5b8ce739af7d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:42.878236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f193d7-d9a9-4f4c-b24f-5b8ce739af7d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:42.878517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f193d7-d9a9-4f4c-b24f-5b8ce739af7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:42.879426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f193d7-d9a9-4f4c-b24f-5b8ce739af7d None admin] RBAC: Adding query filter params (role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=7c35c69c19fd4f4cbc44dd22b88228ac, user.id=0a40ced9f6d3493d817d4b9d965ce69b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:42.879746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f193d7-d9a9-4f4c-b24f-5b8ce739af7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:42.881339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f193d7-d9a9-4f4c-b24f-5b8ce739af7d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:42.888281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 48/97] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:42 2022] GET /identity/v3/role_assignments?user.id=0a40ced9f6d3493d817d4b9d965ce69b&scope.project.id=7c35c69c19fd4f4cbc44dd22b88228ac&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:44.885860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60424422-0a94-4e88-a835-be2ff90b7d82 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:44.886349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60424422-0a94-4e88-a835-be2ff90b7d82 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:44.886349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60424422-0a94-4e88-a835-be2ff90b7d82 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:44.887690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 50/98] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:44.892974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540a69f-ac98-44fd-a5bd-a56ecaef5010 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:44.893396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540a69f-ac98-44fd-a5bd-a56ecaef5010 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:44.893556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540a69f-ac98-44fd-a5bd-a56ecaef5010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:44.910722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a540a69f-ac98-44fd-a5bd-a56ecaef5010 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:44.911917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a540a69f-ac98-44fd-a5bd-a56ecaef5010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:44.987681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:44.995968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 49/99] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:45.017690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60424422-0a94-4e88-a835-be2ff90b7d82 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:45.128971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60424422-0a94-4e88-a835-be2ff90b7d82 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:45.170479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef5641-904f-4e02-8830-c7009daf0b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:45.171196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef5641-904f-4e02-8830-c7009daf0b6f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:45.171480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef5641-904f-4e02-8830-c7009daf0b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:45.171758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef5641-904f-4e02-8830-c7009daf0b6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:45.182933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef5641-904f-4e02-8830-c7009daf0b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7eef5641-904f-4e02-8830-c7009daf0b6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:45.190499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:45.191363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:45.192793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 51/100] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:45.198147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540a69f-ac98-44fd-a5bd-a56ecaef5010 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:45.304966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540a69f-ac98-44fd-a5bd-a56ecaef5010 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:45.345511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d444a7-f6ec-4326-a442-62c8be78ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:45.346244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d444a7-f6ec-4326-a442-62c8be78ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:45.346541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d444a7-f6ec-4326-a442-62c8be78ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:45.346816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d444a7-f6ec-4326-a442-62c8be78ecb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:45.347644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d444a7-f6ec-4326-a442-62c8be78ecb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:45.347952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d444a7-f6ec-4326-a442-62c8be78ecb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:45.349497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d444a7-f6ec-4326-a442-62c8be78ecb4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:45.359363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 50/101] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:45.366215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef5641-904f-4e02-8830-c7009daf0b6f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:45.412574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:45.422381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11e2c-4b94-47a7-b253-723cd3fabf1f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:45.422983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11e2c-4b94-47a7-b253-723cd3fabf1f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:45.423571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb11e2c-4b94-47a7-b253-723cd3fabf1f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:45.425694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 51/102] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:45.430911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2afb-cd8f-428e-ac64-fce626fd4217 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:45.431540 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2afb-cd8f-428e-ac64-fce626fd4217 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:45.432056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f2afb-cd8f-428e-ac64-fce626fd4217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:45.462425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c63f2afb-cd8f-428e-ac64-fce626fd4217 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:45.464017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c63f2afb-cd8f-428e-ac64-fce626fd4217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:45.530164 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef5641-904f-4e02-8830-c7009daf0b6f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:45.536094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:45.566565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 52/103] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:08:45.568122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b25f8-9c96-45e9-8c68-1ab37ed36272 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:45.568411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b25f8-9c96-45e9-8c68-1ab37ed36272 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:45.568687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b25f8-9c96-45e9-8c68-1ab37ed36272 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:45.570211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 53/104] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:45.576049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225b49d-c252-4b4d-9caf-62e40fbbb4bd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:45.576339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225b49d-c252-4b4d-9caf-62e40fbbb4bd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:45.576615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225b49d-c252-4b4d-9caf-62e40fbbb4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:45.589344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c985b4-cdb8-49b3-bc61-049bc8ea0111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:45.590238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c985b4-cdb8-49b3-bc61-049bc8ea0111 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:45.590518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c985b4-cdb8-49b3-bc61-049bc8ea0111 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:45.590792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c985b4-cdb8-49b3-bc61-049bc8ea0111 None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:45.592690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c985b4-cdb8-49b3-bc61-049bc8ea0111 None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:45.594218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c985b4-cdb8-49b3-bc61-049bc8ea0111 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:45.597030 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 52/105] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:45 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:45.602409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c985b4-cdb8-49b3-bc61-049bc8ea0111 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:45.616768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8225b49d-c252-4b4d-9caf-62e40fbbb4bd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:45.617986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8225b49d-c252-4b4d-9caf-62e40fbbb4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:45.745287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 54/106] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:08:45.746210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225b49d-c252-4b4d-9caf-62e40fbbb4bd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:45.775610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:45.786874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c985b4-cdb8-49b3-bc61-049bc8ea0111 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:45.849458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353e09-54c6-464a-96f1-f3d87e1e3ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:45.850346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353e09-54c6-464a-96f1-f3d87e1e3ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:45.850630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353e09-54c6-464a-96f1-f3d87e1e3ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:45.850955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353e09-54c6-464a-96f1-f3d87e1e3ef5 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:45.852196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353e09-54c6-464a-96f1-f3d87e1e3ef5 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:45.853721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353e09-54c6-464a-96f1-f3d87e1e3ef5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:45.855490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 53/107] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:45 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:45.861290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353e09-54c6-464a-96f1-f3d87e1e3ef5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:45.962985 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225b49d-c252-4b4d-9caf-62e40fbbb4bd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:45.982895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:46.005538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340843f-4ed9-4b58-abe4-66b826c173d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.006323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340843f-4ed9-4b58-abe4-66b826c173d3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.006616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340843f-4ed9-4b58-abe4-66b826c173d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.006901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340843f-4ed9-4b58-abe4-66b826c173d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.012608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353e09-54c6-464a-96f1-f3d87e1e3ef5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:46.020404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d340843f-4ed9-4b58-abe4-66b826c173d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d340843f-4ed9-4b58-abe4-66b826c173d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:46.030097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:46.036402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 55/108] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:08:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:46.037373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340843f-4ed9-4b58-abe4-66b826c173d3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.056727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0cc25-346e-4044-ab4a-38abc3491e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.057696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0cc25-346e-4044-ab4a-38abc3491e34 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.058051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0cc25-346e-4044-ab4a-38abc3491e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.058346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0cc25-346e-4044-ab4a-38abc3491e34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.059550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0cc25-346e-4044-ab4a-38abc3491e34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:46.061025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0cc25-346e-4044-ab4a-38abc3491e34 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:46.062892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 54/109] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:46.068856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138d4cc-44b1-46e0-82b0-1e39f2a6e8c7 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.069153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138d4cc-44b1-46e0-82b0-1e39f2a6e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.069453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138d4cc-44b1-46e0-82b0-1e39f2a6e8c7 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.071068 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 55/110] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:46 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:46.073478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405b6e7-ad86-40ec-a26f-3700821e2920 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.073478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405b6e7-ad86-40ec-a26f-3700821e2920 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.073478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405b6e7-ad86-40ec-a26f-3700821e2920 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.074839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 56/111] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:46.079666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405b6e7-ad86-40ec-a26f-3700821e2920 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.108494 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:46.160330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d340843f-4ed9-4b58-abe4-66b826c173d3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:46.184416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405b6e7-ad86-40ec-a26f-3700821e2920 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:46.197021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5377691-8fde-4d88-a5c5-4e6ae6c39de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.198015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377691-8fde-4d88-a5c5-4e6ae6c39de7 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.198307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377691-8fde-4d88-a5c5-4e6ae6c39de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.198589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5377691-8fde-4d88-a5c5-4e6ae6c39de7 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/users/0a40ced9f6d3493d817d4b9d965ce69b/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.200605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5377691-8fde-4d88-a5c5-4e6ae6c39de7 None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd72df2adb4ac4a5cc614f9d1b9167, user_id=0a40ced9f6d3493d817d4b9d965ce69b, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:46.202155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5377691-8fde-4d88-a5c5-4e6ae6c39de7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:46.216262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 56/112] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:08:46 2022] PUT /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/users/0a40ced9f6d3493d817d4b9d965ce69b/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:08:46.217169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5377691-8fde-4d88-a5c5-4e6ae6c39de7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.219762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74572be-8fe0-4ac4-abce-8a8425c06396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.220455 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74572be-8fe0-4ac4-abce-8a8425c06396 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.221707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74572be-8fe0-4ac4-abce-8a8425c06396 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.221707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74572be-8fe0-4ac4-abce-8a8425c06396 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.221990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74572be-8fe0-4ac4-abce-8a8425c06396 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:46.222149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74572be-8fe0-4ac4-abce-8a8425c06396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:46.223592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74572be-8fe0-4ac4-abce-8a8425c06396 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:46.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 57/113] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:08:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:46.240819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c13be-2491-4c33-afb1-4d9602868e1e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.241125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c13be-2491-4c33-afb1-4d9602868e1e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.241523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c13be-2491-4c33-afb1-4d9602868e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.262189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8f6c13be-2491-4c33-afb1-4d9602868e1e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:46.263352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8f6c13be-2491-4c33-afb1-4d9602868e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:46.264062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:46.340605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5377691-8fde-4d88-a5c5-4e6ae6c39de7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:46.347569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 58/114] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:46.348852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5f9cc-a986-4459-ac94-7a4275308904 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.349125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5f9cc-a986-4459-ac94-7a4275308904 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.349492 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5f9cc-a986-4459-ac94-7a4275308904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.375373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e4f5f9cc-a986-4459-ac94-7a4275308904 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:46.376789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e4f5f9cc-a986-4459-ac94-7a4275308904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:46.385700 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241529aa-b428-4085-ab42-e7a7173ca554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.386518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241529aa-b428-4085-ab42-e7a7173ca554 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.386923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241529aa-b428-4085-ab42-e7a7173ca554 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.387304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241529aa-b428-4085-ab42-e7a7173ca554 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.388139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241529aa-b428-4085-ab42-e7a7173ca554 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:46.389755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241529aa-b428-4085-ab42-e7a7173ca554 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:46.399660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:46.430301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 57/115] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:08:46 2022] POST /identity/v3/projects => generated 302 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:46.431854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61ff27-e772-44af-9428-86f3981c6795 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.432870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61ff27-e772-44af-9428-86f3981c6795 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.433354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61ff27-e772-44af-9428-86f3981c6795 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.435345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 58/116] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:46.437232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61ff27-e772-44af-9428-86f3981c6795 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.479456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 59/117] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:46.480494 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5f9cc-a986-4459-ac94-7a4275308904 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.568049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:46.589846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61ff27-e772-44af-9428-86f3981c6795 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:46.616562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5f9cc-a986-4459-ac94-7a4275308904 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:46.639085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b5be5-20c7-4a7d-9b2b-5ee4c5e62754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.640036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b5be5-20c7-4a7d-9b2b-5ee4c5e62754 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.640325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b5be5-20c7-4a7d-9b2b-5ee4c5e62754 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.640599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b5be5-20c7-4a7d-9b2b-5ee4c5e62754 None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.642769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b5be5-20c7-4a7d-9b2b-5ee4c5e62754 None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:46.644183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b5be5-20c7-4a7d-9b2b-5ee4c5e62754 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:46.647001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 59/118] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:46 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:46.648565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7f75e-2305-484f-8666-4344ac7a5c7a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.648565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7f75e-2305-484f-8666-4344ac7a5c7a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.648565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7f75e-2305-484f-8666-4344ac7a5c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.667014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e4e7f75e-2305-484f-8666-4344ac7a5c7a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:46.668248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e4e7f75e-2305-484f-8666-4344ac7a5c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:46.671706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def63af-1cba-4eaa-9c75-ee125c1c90bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.673632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def63af-1cba-4eaa-9c75-ee125c1c90bb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.673923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def63af-1cba-4eaa-9c75-ee125c1c90bb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.674208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def63af-1cba-4eaa-9c75-ee125c1c90bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.689691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def63af-1cba-4eaa-9c75-ee125c1c90bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-6def63af-1cba-4eaa-9c75-ee125c1c90bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:46.700047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:46.700999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:46.701841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 60/119] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:46.702504 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def63af-1cba-4eaa-9c75-ee125c1c90bb None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.722394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:46.753854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 60/120] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:46.755028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7f75e-2305-484f-8666-4344ac7a5c7a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.817196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def63af-1cba-4eaa-9c75-ee125c1c90bb None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:46.853412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e6f3f-8e69-4408-8b7c-705e890b2bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.854142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e6f3f-8e69-4408-8b7c-705e890b2bad None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.854437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e6f3f-8e69-4408-8b7c-705e890b2bad None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.855026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:46.855140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e6f3f-8e69-4408-8b7c-705e890b2bad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.864761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7f75e-2305-484f-8666-4344ac7a5c7a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:46.866301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e6f3f-8e69-4408-8b7c-705e890b2bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8d9e6f3f-8e69-4408-8b7c-705e890b2bad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:46.874359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:46.875263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:46.876033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 61/121] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:46.876708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e6f3f-8e69-4408-8b7c-705e890b2bad None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.905802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc36088-b3af-41a6-9df1-ba2aadcc8ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:46.906940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36088-b3af-41a6-9df1-ba2aadcc8ace None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:46.907370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36088-b3af-41a6-9df1-ba2aadcc8ace None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:46.907974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc36088-b3af-41a6-9df1-ba2aadcc8ace None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:46.909511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc36088-b3af-41a6-9df1-ba2aadcc8ace None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:46.911231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc36088-b3af-41a6-9df1-ba2aadcc8ace None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:46.913867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 61/122] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:46 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:46.915327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc36088-b3af-41a6-9df1-ba2aadcc8ace None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:46.965359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:47.006755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e6f3f-8e69-4408-8b7c-705e890b2bad None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.046480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc36088-b3af-41a6-9df1-ba2aadcc8ace None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.049584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.050817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.051327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.051827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.052800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:47.053350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:47.055690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:47.065947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 62/123] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:47.067160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:47.083928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1398f-8596-4b20-8cb9-c98e91e82e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.084848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1398f-8596-4b20-8cb9-c98e91e82e2f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.085581 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1398f-8596-4b20-8cb9-c98e91e82e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.086286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1398f-8596-4b20-8cb9-c98e91e82e2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.097859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1398f-8596-4b20-8cb9-c98e91e82e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-00b1398f-8596-4b20-8cb9-c98e91e82e2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:47.105955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:47.106832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:47.107631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 62/124] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:08:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:47.108314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1398f-8596-4b20-8cb9-c98e91e82e2f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:47.133057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:47.186151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075ed5f-b6b7-4701-b7db-2a35bc9d3b92 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.220046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:47.225659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1398f-8596-4b20-8cb9-c98e91e82e2f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.227228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.228253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.228837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.229378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.230368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:47.230883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:47.232504 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:47.242721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 63/125] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:47.244097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:47.262464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad4bed-9f3e-4c44-9238-4af959990a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.263422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4bed-9f3e-4c44-9238-4af959990a76 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.263678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4bed-9f3e-4c44-9238-4af959990a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.264029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4bed-9f3e-4c44-9238-4af959990a76 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/users/0a40ced9f6d3493d817d4b9d965ce69b/roles/774686ef0c4c42ba97f56e75b67d313e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.266656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad4bed-9f3e-4c44-9238-4af959990a76 None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd72df2adb4ac4a5cc614f9d1b9167, user_id=0a40ced9f6d3493d817d4b9d965ce69b, role_id=774686ef0c4c42ba97f56e75b67d313e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:47.268009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad4bed-9f3e-4c44-9238-4af959990a76 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:47.286049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 63/126] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:08:47 2022] PUT /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/users/0a40ced9f6d3493d817d4b9d965ce69b/roles/774686ef0c4c42ba97f56e75b67d313e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:08:47.292526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad4bed-9f3e-4c44-9238-4af959990a76 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:47.402806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:47.435639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad4bed-9f3e-4c44-9238-4af959990a76 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.450336 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a9c51-51ad-4c43-80c1-678d146b3d58 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.475009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.475920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.476370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.476922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.477703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:47.478139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:47.479779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:47.492558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 64/127] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:47.494478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:47.525232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b931f2-ef87-4306-857a-eb48de41042d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.526254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b931f2-ef87-4306-857a-eb48de41042d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.526547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b931f2-ef87-4306-857a-eb48de41042d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.526926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b931f2-ef87-4306-857a-eb48de41042d None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.530039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b931f2-ef87-4306-857a-eb48de41042d None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:47.531541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b931f2-ef87-4306-857a-eb48de41042d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:47.537868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 64/128] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:47 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:47.538769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b931f2-ef87-4306-857a-eb48de41042d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:47.615287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d141ecec-9175-4702-b2cf-50b6b59ce032 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.673288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcd3e3-feb2-422a-a05b-a903d2ac49b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.673288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd3e3-feb2-422a-a05b-a903d2ac49b2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.673288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd3e3-feb2-422a-a05b-a903d2ac49b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.673288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcd3e3-feb2-422a-a05b-a903d2ac49b2 None admin] PATH_INFO: `/v3/users/daad4a1e651f40c4a8e915f53c3d333a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.681813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:47.692865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcd3e3-feb2-422a-a05b-a903d2ac49b2 None admin] RBAC: Authorizing `identity:get_user(user_id=daad4a1e651f40c4a8e915f53c3d333a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:47.694889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bcd3e3-feb2-422a-a05b-a903d2ac49b2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:47.696951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 65/129] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:47 2022] GET /identity/v3/users/daad4a1e651f40c4a8e915f53c3d333a => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:47.698119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcd3e3-feb2-422a-a05b-a903d2ac49b2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:47.701300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:47.788016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b931f2-ef87-4306-857a-eb48de41042d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.849317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcd3e3-feb2-422a-a05b-a903d2ac49b2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:47.887411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b550f1a-b937-474c-98a9-090f0e55e405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.888347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550f1a-b937-474c-98a9-090f0e55e405 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.888643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550f1a-b937-474c-98a9-090f0e55e405 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.888922 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b550f1a-b937-474c-98a9-090f0e55e405 None admin] PATH_INFO: `/v3/users/918e7fd9b64348e38ccb56232b8e098a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.891135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b550f1a-b937-474c-98a9-090f0e55e405 None admin] RBAC: Authorizing `identity:get_user(user_id=918e7fd9b64348e38ccb56232b8e098a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:47.892654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b550f1a-b937-474c-98a9-090f0e55e405 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:47.894671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1e4e5-a69e-48b7-b873-7daf5632e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:47.895580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1e4e5-a69e-48b7-b873-7daf5632e488 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:47.895867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1e4e5-a69e-48b7-b873-7daf5632e488 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:47.896149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1e4e5-a69e-48b7-b873-7daf5632e488 None admin] PATH_INFO: `/v3/projects/7c35c69c19fd4f4cbc44dd22b88228ac` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:47.899492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1e4e5-a69e-48b7-b873-7daf5632e488 None admin] RBAC: Authorizing `identity:get_project(project_id=7c35c69c19fd4f4cbc44dd22b88228ac)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:47.901061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1e4e5-a69e-48b7-b873-7daf5632e488 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:47.903095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 66/130] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:47 2022] GET /identity/v3/projects/7c35c69c19fd4f4cbc44dd22b88228ac => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:47.903758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1e4e5-a69e-48b7-b873-7daf5632e488 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:47.904370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 65/131] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:47 2022] GET /identity/v3/users/918e7fd9b64348e38ccb56232b8e098a => generated 247 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:47.906763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b550f1a-b937-474c-98a9-090f0e55e405 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:48.005560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:48.037749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1e4e5-a69e-48b7-b873-7daf5632e488 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:48.076773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb987a35-1aae-48fa-8470-e6f67029f5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:48.078349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987a35-1aae-48fa-8470-e6f67029f5af None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:48.081454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987a35-1aae-48fa-8470-e6f67029f5af None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:48.081744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb987a35-1aae-48fa-8470-e6f67029f5af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:48.107980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b550f1a-b937-474c-98a9-090f0e55e405 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:48.148594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb229dc-c9b4-4afa-ba94-86b591b854c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:48.149614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb229dc-c9b4-4afa-ba94-86b591b854c4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:48.149955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb229dc-c9b4-4afa-ba94-86b591b854c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:48.150251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb229dc-c9b4-4afa-ba94-86b591b854c4 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:48.152094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb229dc-c9b4-4afa-ba94-86b591b854c4 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:48.153655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb229dc-c9b4-4afa-ba94-86b591b854c4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:48.155508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 66/132] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:47 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:48.156285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb229dc-c9b4-4afa-ba94-86b591b854c4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:48.164648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:48.202137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb987a35-1aae-48fa-8470-e6f67029f5af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:48.203897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb987a35-1aae-48fa-8470-e6f67029f5af None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:48.269731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:48.312768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 67/133] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:48.313895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb987a35-1aae-48fa-8470-e6f67029f5af None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:48.392960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb229dc-c9b4-4afa-ba94-86b591b854c4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:48.418826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb987a35-1aae-48fa-8470-e6f67029f5af None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:48.456677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46cb4-51bf-4a66-9d76-6ebd229c28f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:48.457922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46cb4-51bf-4a66-9d76-6ebd229c28f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:48.458348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46cb4-51bf-4a66-9d76-6ebd229c28f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:48.458801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a46cb4-51bf-4a66-9d76-6ebd229c28f3 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/users/918e7fd9b64348e38ccb56232b8e098a/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:48.461565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a46cb4-51bf-4a66-9d76-6ebd229c28f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd72df2adb4ac4a5cc614f9d1b9167, user_id=918e7fd9b64348e38ccb56232b8e098a, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:48.463305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe6fde-a2c3-492c-961e-8e3767ca1e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:48.463582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a46cb4-51bf-4a66-9d76-6ebd229c28f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:48.464425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6fde-a2c3-492c-961e-8e3767ca1e0e None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:48.464810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6fde-a2c3-492c-961e-8e3767ca1e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:48.465204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe6fde-a2c3-492c-961e-8e3767ca1e0e None admin] PATH_INFO: `/v3/projects/7c35c69c19fd4f4cbc44dd22b88228ac/users/0a40ced9f6d3493d817d4b9d965ce69b/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:48.470281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe6fde-a2c3-492c-961e-8e3767ca1e0e None admin] RBAC: Authorizing `identity:create_grant(project_id=7c35c69c19fd4f4cbc44dd22b88228ac, user_id=0a40ced9f6d3493d817d4b9d965ce69b, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:48.472101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfe6fde-a2c3-492c-961e-8e3767ca1e0e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:48.483550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 68/134] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:08:48 2022] PUT /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/users/918e7fd9b64348e38ccb56232b8e098a/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:08:48.485145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46cb4-51bf-4a66-9d76-6ebd229c28f3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:48.503524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 67/135] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:08:48 2022] PUT /identity/v3/projects/7c35c69c19fd4f4cbc44dd22b88228ac/users/0a40ced9f6d3493d817d4b9d965ce69b/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:08:48.534700 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:48.636552 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a46cb4-51bf-4a66-9d76-6ebd229c28f3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:48.646777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283148f-de37-45a6-8909-1271b0381a15 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:48.647243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283148f-de37-45a6-8909-1271b0381a15 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:48.647665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8283148f-de37-45a6-8909-1271b0381a15 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:48.649535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 68/136] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:48.655729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1650ad-5554-4781-a0a9-b0e51c63c09c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:48.656047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1650ad-5554-4781-a0a9-b0e51c63c09c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:48.656409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1650ad-5554-4781-a0a9-b0e51c63c09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:48.679473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100aaed5-cf1a-41be-bc7e-3e414b135f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:48.680774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100aaed5-cf1a-41be-bc7e-3e414b135f73 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:48.681263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100aaed5-cf1a-41be-bc7e-3e414b135f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:48.683158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100aaed5-cf1a-41be-bc7e-3e414b135f73 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:48.684786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100aaed5-cf1a-41be-bc7e-3e414b135f73 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:48.686752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100aaed5-cf1a-41be-bc7e-3e414b135f73 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:48.691249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 69/137] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:48 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:48.693481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8a1650ad-5554-4781-a0a9-b0e51c63c09c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:48.695026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8a1650ad-5554-4781-a0a9-b0e51c63c09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:48.697630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100aaed5-cf1a-41be-bc7e-3e414b135f73 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:48.817950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:48.851239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 69/138] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:08:48.873576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:48.885952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100aaed5-cf1a-41be-bc7e-3e414b135f73 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:48.893222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1650ad-5554-4781-a0a9-b0e51c63c09c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:48.936393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:48.937075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:48.937392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:48.937679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:48.938537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] RBAC: Adding query filter params (role.id=70ff25a62a5149a98cdf09458ee3fa10, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167, user.id=daad4a1e651f40c4a8e915f53c3d333a) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:48.938852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:48.940323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:48.948198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 70/139] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:48 2022] GET /identity/v3/role_assignments?user.id=daad4a1e651f40c4a8e915f53c3d333a&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=70ff25a62a5149a98cdf09458ee3fa10 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:08:49.019847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1650ad-5554-4781-a0a9-b0e51c63c09c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:49.069997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e1870-1c57-474d-987d-84d2eae57085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:49.070759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1870-1c57-474d-987d-84d2eae57085 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:49.071047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1870-1c57-474d-987d-84d2eae57085 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:49.071328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1870-1c57-474d-987d-84d2eae57085 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:49.090052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e1870-1c57-474d-987d-84d2eae57085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-4c0e1870-1c57-474d-987d-84d2eae57085 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:49.098235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:49.099169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:49.099955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 70/140] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:08:49.107278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:49.237562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998f751-5716-4d02-8454-87fe89beffa2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:49.283994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b60f6-8b23-4800-a358-e36e2e552529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:49.284693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b60f6-8b23-4800-a358-e36e2e552529 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:49.284980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b60f6-8b23-4800-a358-e36e2e552529 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:49.285282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b60f6-8b23-4800-a358-e36e2e552529 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:49.286135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b60f6-8b23-4800-a358-e36e2e552529 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:49.286447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b60f6-8b23-4800-a358-e36e2e552529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:49.287957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b60f6-8b23-4800-a358-e36e2e552529 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:49.298087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 71/141] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:49.306387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e1870-1c57-474d-987d-84d2eae57085 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:49.344129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd16bbe-5150-4ea0-939d-4f1824e2929e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:49.344411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd16bbe-5150-4ea0-939d-4f1824e2929e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:49.344664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd16bbe-5150-4ea0-939d-4f1824e2929e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:49.346097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 72/142] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:08:49.355083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ab91-c742-4315-b782-7daf8efcf83d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:49.355379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ab91-c742-4315-b782-7daf8efcf83d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:49.355656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0ab91-c742-4315-b782-7daf8efcf83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:49.378577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eaa0ab91-c742-4315-b782-7daf8efcf83d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:49.379778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eaa0ab91-c742-4315-b782-7daf8efcf83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:49.391774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:49.407506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:49.484189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e1870-1c57-474d-987d-84d2eae57085 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:49.513215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 73/143] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:08:49.537375 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1b3a9-14bc-4d61-abef-39d0a78361af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:49.538376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b3a9-14bc-4d61-abef-39d0a78361af None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:49.538661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b3a9-14bc-4d61-abef-39d0a78361af None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:49.538940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1b3a9-14bc-4d61-abef-39d0a78361af None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:49.540193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1b3a9-14bc-4d61-abef-39d0a78361af None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:49.541801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1b3a9-14bc-4d61-abef-39d0a78361af None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:49.543861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 71/144] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:49 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:49.549258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0ab91-c742-4315-b782-7daf8efcf83d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:49.562282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1b3a9-14bc-4d61-abef-39d0a78361af None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:49.797951 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1b3a9-14bc-4d61-abef-39d0a78361af None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:49.846920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0ab91-c742-4315-b782-7daf8efcf83d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:49.849064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:49.900112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d86ac-38d8-4ff5-be9e-9d94908b2345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:49.901391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d86ac-38d8-4ff5-be9e-9d94908b2345 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:49.901892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d86ac-38d8-4ff5-be9e-9d94908b2345 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:49.902434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d86ac-38d8-4ff5-be9e-9d94908b2345 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:49.904093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d86ac-38d8-4ff5-be9e-9d94908b2345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:49.907346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d86ac-38d8-4ff5-be9e-9d94908b2345 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:49.910283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 72/145] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:49.911957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5520f-f905-4300-93e4-3fd58aed60bb None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:49.912503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5520f-f905-4300-93e4-3fd58aed60bb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:49.913018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5520f-f905-4300-93e4-3fd58aed60bb None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:49.915249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 73/146] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:49.916350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5520f-f905-4300-93e4-3fd58aed60bb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:49.926575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb3268-c783-45b9-a925-2c4c22bd6b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:49.927261 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb3268-c783-45b9-a925-2c4c22bd6b70 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:49.927549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb3268-c783-45b9-a925-2c4c22bd6b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:49.927832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb3268-c783-45b9-a925-2c4c22bd6b70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:49.967514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:50.077091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5520f-f905-4300-93e4-3fd58aed60bb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:50.082276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb3268-c783-45b9-a925-2c4c22bd6b70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:50.083867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb3268-c783-45b9-a925-2c4c22bd6b70 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:50.117057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4025e0b-4c08-4241-b97b-af90d9a96c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:50.118293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4025e0b-4c08-4241-b97b-af90d9a96c42 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:50.118779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4025e0b-4c08-4241-b97b-af90d9a96c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:50.119342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4025e0b-4c08-4241-b97b-af90d9a96c42 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:50.120294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4025e0b-4c08-4241-b97b-af90d9a96c42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:50.122215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4025e0b-4c08-4241-b97b-af90d9a96c42 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:50.150792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 74/147] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:08:49 2022] POST /identity/v3/users => generated 283 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:50.152271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b70103-d092-45ed-8dd0-e5c87544127b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:50.152572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b70103-d092-45ed-8dd0-e5c87544127b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:50.152852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b70103-d092-45ed-8dd0-e5c87544127b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:50.211074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e5b70103-d092-45ed-8dd0-e5c87544127b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:50.212254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e5b70103-d092-45ed-8dd0-e5c87544127b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:50.249511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 74/148] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:50.255644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb3268-c783-45b9-a925-2c4c22bd6b70 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:50.319292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:50.324159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:50.386764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb3268-c783-45b9-a925-2c4c22bd6b70 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:50.429664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddd7d5-5bb7-4d33-8038-eb41a47571a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:50.431023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddd7d5-5bb7-4d33-8038-eb41a47571a4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:50.431577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddd7d5-5bb7-4d33-8038-eb41a47571a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:50.432087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecddd7d5-5bb7-4d33-8038-eb41a47571a4 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:50.434108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddd7d5-5bb7-4d33-8038-eb41a47571a4 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:50.435854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecddd7d5-5bb7-4d33-8038-eb41a47571a4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:50.437914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 75/149] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:50 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:08:50.443825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 75/150] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:50.450970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddd7d5-5bb7-4d33-8038-eb41a47571a4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:50.487113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b70103-d092-45ed-8dd0-e5c87544127b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:50.597668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecddd7d5-5bb7-4d33-8038-eb41a47571a4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:50.632396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:50.670607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:50.671653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:50.672902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:50.673486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:50.674538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] RBAC: Adding query filter params (role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167, user.id=0a40ced9f6d3493d817d4b9d965ce69b) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:50.675070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:50.676729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:50.686177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 76/151] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:50 2022] GET /identity/v3/role_assignments?user.id=0a40ced9f6d3493d817d4b9d965ce69b&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:50.712253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b70103-d092-45ed-8dd0-e5c87544127b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:50.779000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d59b5-2777-4f85-b235-87fc05fda04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:50.779848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d59b5-2777-4f85-b235-87fc05fda04a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:50.780121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d59b5-2777-4f85-b235-87fc05fda04a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:50.780397 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d59b5-2777-4f85-b235-87fc05fda04a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:50.798231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d59b5-2777-4f85-b235-87fc05fda04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5d9d59b5-2777-4f85-b235-87fc05fda04a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:50.806696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:50.807694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:50.808470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 76/152] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:08:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:50.815284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:50.965428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217d63f-01f1-4915-a3ae-91d072362d2d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:51.003967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:51.004645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:51.004927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:51.005218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:51.005980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:51.006283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:51.007788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:51.019761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 77/153] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:08:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:51.025315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d59b5-2777-4f85-b235-87fc05fda04a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:51.072075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:51.157061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d59b5-2777-4f85-b235-87fc05fda04a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:51.212753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9341bf-3868-4bae-9bd1-76bdac3bd6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:51.214301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9341bf-3868-4bae-9bd1-76bdac3bd6e0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:51.214754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9341bf-3868-4bae-9bd1-76bdac3bd6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:51.215251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9341bf-3868-4bae-9bd1-76bdac3bd6e0 None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:51.217270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9341bf-3868-4bae-9bd1-76bdac3bd6e0 None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:51.218992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9341bf-3868-4bae-9bd1-76bdac3bd6e0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:51.220878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 77/154] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:51 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:51.226656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:51.279289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:51.346541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750e412c-1a94-4f15-ba28-ba05f89b5182 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:51.389597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9648b2a9-79e9-47e0-8899-ced9883aa3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:51.390615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b2a9-79e9-47e0-8899-ced9883aa3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:51.391146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b2a9-79e9-47e0-8899-ced9883aa3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:51.391598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9648b2a9-79e9-47e0-8899-ced9883aa3e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:51.479474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9648b2a9-79e9-47e0-8899-ced9883aa3e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:51.480937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9648b2a9-79e9-47e0-8899-ced9883aa3e2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:51.541667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:51.614999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 78/155] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:51.622333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9341bf-3868-4bae-9bd1-76bdac3bd6e0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:51.763015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:51.772083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9341bf-3868-4bae-9bd1-76bdac3bd6e0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:51.811763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb83023-d4e6-4900-b178-051e4b5b7b86 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:51.812057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb83023-d4e6-4900-b178-051e4b5b7b86 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:51.812666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb83023-d4e6-4900-b178-051e4b5b7b86 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:51.814428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 79/156] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:51.815875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748c9e7-8534-4bdd-9bfd-937d3eb675c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:51.816767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c9e7-8534-4bdd-9bfd-937d3eb675c2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:51.817052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c9e7-8534-4bdd-9bfd-937d3eb675c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:51.817359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748c9e7-8534-4bdd-9bfd-937d3eb675c2 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:51.818597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748c9e7-8534-4bdd-9bfd-937d3eb675c2 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:51.820415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748c9e7-8534-4bdd-9bfd-937d3eb675c2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:51.820975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d23a8-1679-4bf6-a5b0-9289d0edffdc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:51.821485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d23a8-1679-4bf6-a5b0-9289d0edffdc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:51.821981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d23a8-1679-4bf6-a5b0-9289d0edffdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:51.822177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 78/157] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:51 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:51.828041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748c9e7-8534-4bdd-9bfd-937d3eb675c2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:51.863035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-153d23a8-1679-4bf6-a5b0-9289d0edffdc None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:51.864637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-153d23a8-1679-4bf6-a5b0-9289d0edffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:51.970415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 80/158] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 17:08:52.008495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d23a8-1679-4bf6-a5b0-9289d0edffdc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:52.076698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748c9e7-8534-4bdd-9bfd-937d3eb675c2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:52.124635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:52.161338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc6d2f-cac8-430d-bad4-0dd9da732349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:52.162067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc6d2f-cac8-430d-bad4-0dd9da732349 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.162352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc6d2f-cac8-430d-bad4-0dd9da732349 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.162632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc6d2f-cac8-430d-bad4-0dd9da732349 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.163770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc6d2f-cac8-430d-bad4-0dd9da732349 None admin] RBAC: Adding query filter params (role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167, user.id=0a40ced9f6d3493d817d4b9d965ce69b) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:52.164091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc6d2f-cac8-430d-bad4-0dd9da732349 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:52.165635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc6d2f-cac8-430d-bad4-0dd9da732349 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:52.168267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d23a8-1679-4bf6-a5b0-9289d0edffdc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:52.177095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 79/159] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:51 2022] GET /identity/v3/role_assignments?user.id=0a40ced9f6d3493d817d4b9d965ce69b&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 608 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:08:52.225147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23622aa1-07f2-4c42-aaf1-ac5603644d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:52.226241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23622aa1-07f2-4c42-aaf1-ac5603644d1e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.226518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23622aa1-07f2-4c42-aaf1-ac5603644d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.226788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23622aa1-07f2-4c42-aaf1-ac5603644d1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.246870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23622aa1-07f2-4c42-aaf1-ac5603644d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-23622aa1-07f2-4c42-aaf1-ac5603644d1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:52.254611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:52.255565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:52.256357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:52.258929 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1aea-c049-4960-b76d-de3419b44ef6 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.259283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1aea-c049-4960-b76d-de3419b44ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.259572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e1aea-c049-4960-b76d-de3419b44ef6 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.261287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 80/161] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:52.261468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 81/161] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:52.266322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e1aea-c049-4960-b76d-de3419b44ef6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:52.266948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03c66-1639-4e55-aef0-8c68c6210308 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.267242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03c66-1639-4e55-aef0-8c68c6210308 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.267519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c03c66-1639-4e55-aef0-8c68c6210308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.291068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c2c03c66-1639-4e55-aef0-8c68c6210308 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:52.294142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c2c03c66-1639-4e55-aef0-8c68c6210308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:52.337620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:52.458568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e1aea-c049-4960-b76d-de3419b44ef6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:52.488116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 82/162] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:52.489686 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12d3f-79ba-4b60-b992-88a925a96016 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.489981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12d3f-79ba-4b60-b992-88a925a96016 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.490718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc12d3f-79ba-4b60-b992-88a925a96016 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.492019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 83/163] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:52.497770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdb9df-7c1e-4b7d-bbde-7f6f5177d2fd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.498056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdb9df-7c1e-4b7d-bbde-7f6f5177d2fd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.498337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdb9df-7c1e-4b7d-bbde-7f6f5177d2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.526487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8cbdb9df-7c1e-4b7d-bbde-7f6f5177d2fd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:52.528192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8cbdb9df-7c1e-4b7d-bbde-7f6f5177d2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:52.538009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:52.538937 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.539321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.539607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.540394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:52.540707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:52.542186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:52.561006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 81/164] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:52.563042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:52.674243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:52.682253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:52.685974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb97bd-dfa4-4f4f-b23a-e3ee9245e9e2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:52.725601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c981abf-a86e-45d1-bfb7-57dfa8d6f877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:52.726811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981abf-a86e-45d1-bfb7-57dfa8d6f877 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.727287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981abf-a86e-45d1-bfb7-57dfa8d6f877 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.727754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981abf-a86e-45d1-bfb7-57dfa8d6f877 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.727875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 84/165] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:08:52.728775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdb9df-7c1e-4b7d-bbde-7f6f5177d2fd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:52.752074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c981abf-a86e-45d1-bfb7-57dfa8d6f877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8c981abf-a86e-45d1-bfb7-57dfa8d6f877 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:52.766409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:52.767344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:52.768128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:52.771208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 82/166] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:08:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:52.777575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c981abf-a86e-45d1-bfb7-57dfa8d6f877 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:52.885368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c981abf-a86e-45d1-bfb7-57dfa8d6f877 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:52.920861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:52.921965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:52.922445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:52.922912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:52.923964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:52.924529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:52.926147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:52.935201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 83/167] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:08:52.936318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:52.952732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:53.041864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec2f0a-7e1c-4f7f-bbd6-c4239a8bb980 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:53.047096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdb9df-7c1e-4b7d-bbde-7f6f5177d2fd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:53.078916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45613727-dc4f-41a3-b293-f314adfea3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:53.080058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45613727-dc4f-41a3-b293-f314adfea3cf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:53.080535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45613727-dc4f-41a3-b293-f314adfea3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:53.081001 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45613727-dc4f-41a3-b293-f314adfea3cf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:53.092682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45613727-dc4f-41a3-b293-f314adfea3cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-45613727-dc4f-41a3-b293-f314adfea3cf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:53.101232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:53.102126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:08:53.103029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:53.105937 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 84/168] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:53.106951 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45613727-dc4f-41a3-b293-f314adfea3cf None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:53.149920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc9856-a050-4fb4-b8df-c022fa2b61b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:53.150816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc9856-a050-4fb4-b8df-c022fa2b61b3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:53.151101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc9856-a050-4fb4-b8df-c022fa2b61b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:53.151374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc9856-a050-4fb4-b8df-c022fa2b61b3 None admin] PATH_INFO: `/v3/users/0a40ced9f6d3493d817d4b9d965ce69b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:53.152628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc9856-a050-4fb4-b8df-c022fa2b61b3 None admin] RBAC: Authorizing `identity:get_user(user_id=0a40ced9f6d3493d817d4b9d965ce69b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:53.154325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc9856-a050-4fb4-b8df-c022fa2b61b3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:53.156131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 85/169] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:52 2022] GET /identity/v3/users/0a40ced9f6d3493d817d4b9d965ce69b => generated 275 bytes in 428 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:53.159662 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc9856-a050-4fb4-b8df-c022fa2b61b3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:53.200446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:53.227342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45613727-dc4f-41a3-b293-f314adfea3cf None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:53.277868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7822-3c56-4103-946d-b475e622c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:53.278760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7822-3c56-4103-946d-b475e622c76f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:53.279043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7822-3c56-4103-946d-b475e622c76f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:53.279321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee7822-3c56-4103-946d-b475e622c76f None admin] PATH_INFO: `/v3/users/918e7fd9b64348e38ccb56232b8e098a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:53.282844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee7822-3c56-4103-946d-b475e622c76f None admin] RBAC: Authorizing `identity:get_user(user_id=918e7fd9b64348e38ccb56232b8e098a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:53.284344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee7822-3c56-4103-946d-b475e622c76f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:53.286081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 85/170] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:53 2022] GET /identity/v3/users/918e7fd9b64348e38ccb56232b8e098a => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:53.286850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7822-3c56-4103-946d-b475e622c76f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:53.310132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:53.395500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc9856-a050-4fb4-b8df-c022fa2b61b3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:53.480144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:53.480608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:53.480901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:53.481195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:53.481967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:53.482275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:53.483683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:53.507570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 86/171] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:53.508184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:53.511276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee7822-3c56-4103-946d-b475e622c76f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:53.621885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5b178-2008-47b2-9f8b-3dc3e783d8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:53.622919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5b178-2008-47b2-9f8b-3dc3e783d8ee None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:53.623441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5b178-2008-47b2-9f8b-3dc3e783d8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:53.623911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5b178-2008-47b2-9f8b-3dc3e783d8ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:53.641228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb0806-dea6-4820-9178-eab014bbb5a3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:53.651812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:53.675128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:53.689723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdab37-a24a-4c03-b6c9-ffeb5154cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:53.690779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdab37-a24a-4c03-b6c9-ffeb5154cd7a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:53.691277 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdab37-a24a-4c03-b6c9-ffeb5154cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:53.692033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdab37-a24a-4c03-b6c9-ffeb5154cd7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:53.760395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5b178-2008-47b2-9f8b-3dc3e783d8ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:53.762699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5b178-2008-47b2-9f8b-3dc3e783d8ee None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:53.768104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdab37-a24a-4c03-b6c9-ffeb5154cd7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:53.770133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdab37-a24a-4c03-b6c9-ffeb5154cd7a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:53.882144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 86/172] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:53.884876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 87/173] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:53.886410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0c9b-e94a-41ad-97ac-fb46488ab00c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:53.886826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0c9b-e94a-41ad-97ac-fb46488ab00c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:53.887305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b0c9b-e94a-41ad-97ac-fb46488ab00c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:53.887955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5b178-2008-47b2-9f8b-3dc3e783d8ee None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:53.888924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 88/174] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:53.889859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b0c9b-e94a-41ad-97ac-fb46488ab00c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:53.918780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:53.936116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:53.996101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5b178-2008-47b2-9f8b-3dc3e783d8ee None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.011192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b0c9b-e94a-41ad-97ac-fb46488ab00c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.031338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db463e-5dad-4c5d-9ce0-f1850c7e6e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.032538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db463e-5dad-4c5d-9ce0-f1850c7e6e18 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.033013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db463e-5dad-4c5d-9ce0-f1850c7e6e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.033524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db463e-5dad-4c5d-9ce0-f1850c7e6e18 None admin] PATH_INFO: `/v3/users/daad4a1e651f40c4a8e915f53c3d333a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.035022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db463e-5dad-4c5d-9ce0-f1850c7e6e18 None admin] RBAC: Authorizing `identity:get_user(user_id=daad4a1e651f40c4a8e915f53c3d333a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:54.036718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db463e-5dad-4c5d-9ce0-f1850c7e6e18 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:54.039005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 87/175] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:53 2022] GET /identity/v3/users/daad4a1e651f40c4a8e915f53c3d333a => generated 289 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:54.040077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db463e-5dad-4c5d-9ce0-f1850c7e6e18 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.047680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f565300-bb17-444a-a698-1bac2d0cff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.048546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f565300-bb17-444a-a698-1bac2d0cff46 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.048827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f565300-bb17-444a-a698-1bac2d0cff46 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.049099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f565300-bb17-444a-a698-1bac2d0cff46 None admin] PATH_INFO: `/v3/projects/7c35c69c19fd4f4cbc44dd22b88228ac` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.050630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f565300-bb17-444a-a698-1bac2d0cff46 None admin] RBAC: Authorizing `identity:get_project(project_id=7c35c69c19fd4f4cbc44dd22b88228ac)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:54.051849 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f565300-bb17-444a-a698-1bac2d0cff46 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:54.053909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 89/176] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:53 2022] GET /identity/v3/projects/7c35c69c19fd4f4cbc44dd22b88228ac => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:54.058884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24731329-5869-44ee-b91c-ea44b46b02fd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.058884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24731329-5869-44ee-b91c-ea44b46b02fd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.058884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24731329-5869-44ee-b91c-ea44b46b02fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.074372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-24731329-5869-44ee-b91c-ea44b46b02fd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:54.075660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-24731329-5869-44ee-b91c-ea44b46b02fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:54.151876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db463e-5dad-4c5d-9ce0-f1850c7e6e18 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.157251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 90/177] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:54.158235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24731329-5869-44ee-b91c-ea44b46b02fd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.164890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:54.196191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:54.200160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1597350-51ab-4d77-9e6d-224b4dc21f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.201048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597350-51ab-4d77-9e6d-224b4dc21f96 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.201554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597350-51ab-4d77-9e6d-224b4dc21f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.201835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1597350-51ab-4d77-9e6d-224b4dc21f96 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.204237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1597350-51ab-4d77-9e6d-224b4dc21f96 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:54.205798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1597350-51ab-4d77-9e6d-224b4dc21f96 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:54.207642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 88/178] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:54 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:54.208434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1597350-51ab-4d77-9e6d-224b4dc21f96 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.309935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24731329-5869-44ee-b91c-ea44b46b02fd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.318797 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1597350-51ab-4d77-9e6d-224b4dc21f96 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.354169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.354579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361e6e8-d059-422b-9aa4-626bc5f74c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.355043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.355137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.355367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.355457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361e6e8-d059-422b-9aa4-626bc5f74c92 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.355731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361e6e8-d059-422b-9aa4-626bc5f74c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.356003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361e6e8-d059-422b-9aa4-626bc5f74c92 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.356211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] RBAC: Adding query filter params (role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=7c35c69c19fd4f4cbc44dd22b88228ac, user.id=0a40ced9f6d3493d817d4b9d965ce69b) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:54.356530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:54.357270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361e6e8-d059-422b-9aa4-626bc5f74c92 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:54.358026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:54.358711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361e6e8-d059-422b-9aa4-626bc5f74c92 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:54.360536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 91/179] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:54 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:54.361227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361e6e8-d059-422b-9aa4-626bc5f74c92 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.364689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 89/180] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:54 2022] GET /identity/v3/role_assignments?user.id=0a40ced9f6d3493d817d4b9d965ce69b&scope.project.id=7c35c69c19fd4f4cbc44dd22b88228ac&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:54.365384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.415815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:54.464672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:54.482847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361e6e8-d059-422b-9aa4-626bc5f74c92 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.487639 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ebc91-ba98-4593-80fa-8893dbd26dd6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.523640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc15561-cba4-495f-b61b-d21a7fb2c285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.524450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc15561-cba4-495f-b61b-d21a7fb2c285 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.524729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc15561-cba4-495f-b61b-d21a7fb2c285 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.525001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc15561-cba4-495f-b61b-d21a7fb2c285 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.532349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773cb53-d458-416c-9350-13d75e5a7ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.533027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773cb53-d458-416c-9350-13d75e5a7ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.533363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773cb53-d458-416c-9350-13d75e5a7ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.533641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773cb53-d458-416c-9350-13d75e5a7ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.535114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773cb53-d458-416c-9350-13d75e5a7ec7 None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167, user.id=918e7fd9b64348e38ccb56232b8e098a) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:54.535114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773cb53-d458-416c-9350-13d75e5a7ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:54.536270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773cb53-d458-416c-9350-13d75e5a7ec7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:54.537353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc15561-cba4-495f-b61b-d21a7fb2c285 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:54.545270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 90/181] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:54 2022] GET /identity/v3/role_assignments?user.id=918e7fd9b64348e38ccb56232b8e098a&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9fc15561-cba4-495f-b61b-d21a7fb2c285 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:54.546624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:54.547678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:54.548645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:54.549437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25042674-2a1e-4361-af06-911d01a6ec54 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.549437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25042674-2a1e-4361-af06-911d01a6ec54 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.549437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25042674-2a1e-4361-af06-911d01a6ec54 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.549437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 91/182] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:54.550331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 92/183] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:08:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:54.554230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7d323-689c-4d8d-baad-162c20dfef98 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.554230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7d323-689c-4d8d-baad-162c20dfef98 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.555000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7d323-689c-4d8d-baad-162c20dfef98 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.556274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 92/184] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:54.556858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc15561-cba4-495f-b61b-d21a7fb2c285 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.560317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e851dc-2e3f-463c-9003-9e50e258f8bb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.560317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e851dc-2e3f-463c-9003-9e50e258f8bb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.560317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e851dc-2e3f-463c-9003-9e50e258f8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.578463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-06e851dc-2e3f-463c-9003-9e50e258f8bb None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:54.580058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-06e851dc-2e3f-463c-9003-9e50e258f8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:54.677233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 93/185] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:54.681970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e851dc-2e3f-463c-9003-9e50e258f8bb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.694710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc15561-cba4-495f-b61b-d21a7fb2c285 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.729982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:54.735053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91820dd-3c70-4d95-8bb4-fdd4005825ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.736548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91820dd-3c70-4d95-8bb4-fdd4005825ad None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.736929 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91820dd-3c70-4d95-8bb4-fdd4005825ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.737340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91820dd-3c70-4d95-8bb4-fdd4005825ad None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/users/daad4a1e651f40c4a8e915f53c3d333a/roles/70ff25a62a5149a98cdf09458ee3fa10` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.739530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91820dd-3c70-4d95-8bb4-fdd4005825ad None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd72df2adb4ac4a5cc614f9d1b9167, user_id=daad4a1e651f40c4a8e915f53c3d333a, role_id=70ff25a62a5149a98cdf09458ee3fa10)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:54.741154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91820dd-3c70-4d95-8bb4-fdd4005825ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:54.757413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 93/186] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:08:54 2022] PUT /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/users/daad4a1e651f40c4a8e915f53c3d333a/roles/70ff25a62a5149a98cdf09458ee3fa10 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:08:54.758731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83423a-4723-4d5d-a498-aef04cb53a5b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.759021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83423a-4723-4d5d-a498-aef04cb53a5b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.759305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83423a-4723-4d5d-a498-aef04cb53a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.779941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7b83423a-4723-4d5d-a498-aef04cb53a5b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:54.781329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7b83423a-4723-4d5d-a498-aef04cb53a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:54.784159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:54.831722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e851dc-2e3f-463c-9003-9e50e258f8bb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:54.871706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204b581-0306-472e-badb-5784debd7179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:54.872439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204b581-0306-472e-badb-5784debd7179 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:54.872725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204b581-0306-472e-badb-5784debd7179 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:54.872985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204b581-0306-472e-badb-5784debd7179 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:54.873801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204b581-0306-472e-badb-5784debd7179 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:54.874109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204b581-0306-472e-badb-5784debd7179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:54.881429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204b581-0306-472e-badb-5784debd7179 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:54.881622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 94/187] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:54.881622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83423a-4723-4d5d-a498-aef04cb53a5b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.894155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 94/188] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:54.901274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204b581-0306-472e-badb-5784debd7179 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:54.999744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83423a-4723-4d5d-a498-aef04cb53a5b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.009886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:55.014217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204b581-0306-472e-badb-5784debd7179 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.037763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea8c57-25ba-4482-8c77-96ac7f0c2fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.039111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea8c57-25ba-4482-8c77-96ac7f0c2fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.039626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea8c57-25ba-4482-8c77-96ac7f0c2fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.040104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea8c57-25ba-4482-8c77-96ac7f0c2fa7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.047788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:55.050150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d7b61-1fcd-473e-aa1d-280b80f5778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.051242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d7b61-1fcd-473e-aa1d-280b80f5778f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.051430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d7b61-1fcd-473e-aa1d-280b80f5778f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.051752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d7b61-1fcd-473e-aa1d-280b80f5778f None admin] PATH_INFO: `/v3/users/5df3f5a3c0d14010a8317fa68a23c2db` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.052847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea8c57-25ba-4482-8c77-96ac7f0c2fa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-41ea8c57-25ba-4482-8c77-96ac7f0c2fa7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:55.061344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:55.062246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:55.063050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 95/189] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:08:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:55.064382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea8c57-25ba-4482-8c77-96ac7f0c2fa7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:55.071967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d7b61-1fcd-473e-aa1d-280b80f5778f None admin] RBAC: Authorizing `identity:get_user(user_id=5df3f5a3c0d14010a8317fa68a23c2db)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.073875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49d7b61-1fcd-473e-aa1d-280b80f5778f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.077331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 95/190] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:54 2022] GET /identity/v3/users/5df3f5a3c0d14010a8317fa68a23c2db => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:55.078133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d7b61-1fcd-473e-aa1d-280b80f5778f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:55.176131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea8c57-25ba-4482-8c77-96ac7f0c2fa7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.181203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d7b61-1fcd-473e-aa1d-280b80f5778f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.213640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315f7c-28f7-4a9b-bc5b-d4699cf39a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.214444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315f7c-28f7-4a9b-bc5b-d4699cf39a7d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.214728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315f7c-28f7-4a9b-bc5b-d4699cf39a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.215013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315f7c-28f7-4a9b-bc5b-d4699cf39a7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.216177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315f7c-28f7-4a9b-bc5b-d4699cf39a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.217707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d315f7c-28f7-4a9b-bc5b-d4699cf39a7d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.219051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.219346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 96/191] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:55.220074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.220170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315f7c-28f7-4a9b-bc5b-d4699cf39a7d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:55.220444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.220722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.225218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:55.225218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.225218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.232039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 96/192] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:08:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:55.232039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:55.259036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:55.301004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:55.339905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d315f7c-28f7-4a9b-bc5b-d4699cf39a7d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.345865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aca0d-35bc-4665-a2bf-4c48bcfaf65c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.375457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bd483-89ad-456c-a203-a3740014b339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.376120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bd483-89ad-456c-a203-a3740014b339 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.376409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bd483-89ad-456c-a203-a3740014b339 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.376689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bd483-89ad-456c-a203-a3740014b339 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.386127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d98d-17a7-4fa3-b055-11728921791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.387553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d98d-17a7-4fa3-b055-11728921791a None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.388029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d98d-17a7-4fa3-b055-11728921791a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.388492 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a7d98d-17a7-4fa3-b055-11728921791a None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.389872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7d98d-17a7-4fa3-b055-11728921791a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.392159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a7d98d-17a7-4fa3-b055-11728921791a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.414961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 97/193] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:08:55 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:55.425437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d98d-17a7-4fa3-b055-11728921791a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:55.476532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bd483-89ad-456c-a203-a3740014b339 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.478273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bd483-89ad-456c-a203-a3740014b339 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.532904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:55.567913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:55.574217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a7d98d-17a7-4fa3-b055-11728921791a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.621267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 97/194] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:55.627858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bd483-89ad-456c-a203-a3740014b339 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:55.630180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6095c24-06c4-4547-934c-30e37fa5f346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.632195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6095c24-06c4-4547-934c-30e37fa5f346 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.632551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6095c24-06c4-4547-934c-30e37fa5f346 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.633001 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6095c24-06c4-4547-934c-30e37fa5f346 None admin] PATH_INFO: `/v3/users/5df3f5a3c0d14010a8317fa68a23c2db` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.634386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6095c24-06c4-4547-934c-30e37fa5f346 None admin] RBAC: Authorizing `identity:get_user(user_id=5df3f5a3c0d14010a8317fa68a23c2db)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.635956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6095c24-06c4-4547-934c-30e37fa5f346 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.638286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 98/195] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:55 2022] GET /identity/v3/users/5df3f5a3c0d14010a8317fa68a23c2db => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:55.644531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6095c24-06c4-4547-934c-30e37fa5f346 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:55.756585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bd483-89ad-456c-a203-a3740014b339 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.758715 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6095c24-06c4-4547-934c-30e37fa5f346 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.801281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dd2e4-4ec1-43f5-8637-c787830f4437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.801962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dd2e4-4ec1-43f5-8637-c787830f4437 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.802230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dd2e4-4ec1-43f5-8637-c787830f4437 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.802512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1dd2e4-4ec1-43f5-8637-c787830f4437 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.804092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc22043-1b62-4741-9df7-2331258f4c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.805383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22043-1b62-4741-9df7-2331258f4c0b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.805844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22043-1b62-4741-9df7-2331258f4c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.806522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22043-1b62-4741-9df7-2331258f4c0b None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.809024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc22043-1b62-4741-9df7-2331258f4c0b None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.810678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc22043-1b62-4741-9df7-2331258f4c0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.812579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 98/196] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:55 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:55.818609 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:55.821043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc22043-1b62-4741-9df7-2331258f4c0b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:55.861377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:55.880610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1dd2e4-4ec1-43f5-8637-c787830f4437 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.882870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1dd2e4-4ec1-43f5-8637-c787830f4437 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.937562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc22043-1b62-4741-9df7-2331258f4c0b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:55.982192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:55.983274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:55.983701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:55.984273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:55.985698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=fc31ecc9189941fa909edb765d79c496, user.id=5df3f5a3c0d14010a8317fa68a23c2db) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:55.986424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:55.988518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:55.999362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 99/197] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:55 2022] GET /identity/v3/role_assignments?user.id=5df3f5a3c0d14010a8317fa68a23c2db&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:56.015295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 99/198] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:08:56.020771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:56.132885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:56.137804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a9641-12a7-4c06-8072-0e1b049371ff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:56.199100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb460b8d-e09e-4483-a7bc-4bdd7804387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:56.200018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460b8d-e09e-4483-a7bc-4bdd7804387f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:56.200307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460b8d-e09e-4483-a7bc-4bdd7804387f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:56.200590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460b8d-e09e-4483-a7bc-4bdd7804387f None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:56.201973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb460b8d-e09e-4483-a7bc-4bdd7804387f None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:56.203450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb460b8d-e09e-4483-a7bc-4bdd7804387f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:56.205589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 100/199] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:56 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:56.212231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dd2e4-4ec1-43f5-8637-c787830f4437 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:56.259043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:56.318695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1dd2e4-4ec1-43f5-8637-c787830f4437 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:56.354900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea6278-8ff2-4118-a32d-92e8ef9e2aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:56.355565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6278-8ff2-4118-a32d-92e8ef9e2aed None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:56.355848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6278-8ff2-4118-a32d-92e8ef9e2aed None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:56.356126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6278-8ff2-4118-a32d-92e8ef9e2aed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:56.356962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea6278-8ff2-4118-a32d-92e8ef9e2aed None admin] RBAC: Adding query filter params (role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=fc31ecc9189941fa909edb765d79c496, user.id=5df3f5a3c0d14010a8317fa68a23c2db) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:56.357307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea6278-8ff2-4118-a32d-92e8ef9e2aed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:56.358763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea6278-8ff2-4118-a32d-92e8ef9e2aed None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:56.364716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 100/200] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:56 2022] GET /identity/v3/role_assignments?user.id=5df3f5a3c0d14010a8317fa68a23c2db&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:57.173106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2983-1371-48d9-b974-d03f63b949a8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:57.173683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2983-1371-48d9-b974-d03f63b949a8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:57.173683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d2983-1371-48d9-b974-d03f63b949a8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:57.175329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 101/201] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:57.181043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50c87-4eca-40bc-8a01-7fae66435307 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:57.181492 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50c87-4eca-40bc-8a01-7fae66435307 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:57.181680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b50c87-4eca-40bc-8a01-7fae66435307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:57.199225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-09b50c87-4eca-40bc-8a01-7fae66435307 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:57.200465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-09b50c87-4eca-40bc-8a01-7fae66435307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:57.274479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 101/202] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:57.309128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d2983-1371-48d9-b974-d03f63b949a8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:57.450866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d2983-1371-48d9-b974-d03f63b949a8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:57.507576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b48e5d-e3bc-43a8-8535-3abeaf05b225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:57.508402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b48e5d-e3bc-43a8-8535-3abeaf05b225 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:57.508688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b48e5d-e3bc-43a8-8535-3abeaf05b225 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:57.508968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b48e5d-e3bc-43a8-8535-3abeaf05b225 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:57.530065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b48e5d-e3bc-43a8-8535-3abeaf05b225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-30b48e5d-e3bc-43a8-8535-3abeaf05b225 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:57.537889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:57.538931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:57.539739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 102/203] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:57.545210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50c87-4eca-40bc-8a01-7fae66435307 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:57.614290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:57.699875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b50c87-4eca-40bc-8a01-7fae66435307 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:57.737045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:57.737712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:57.737998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:57.738277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:57.739051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:57.739360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:57.740755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:57.751192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 102/204] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:57.756674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b48e5d-e3bc-43a8-8535-3abeaf05b225 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:57.807412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:57.888233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b48e5d-e3bc-43a8-8535-3abeaf05b225 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:57.928816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602bcf-5f66-479f-9ae5-0e03a137cf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:57.929743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602bcf-5f66-479f-9ae5-0e03a137cf3c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:57.930020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602bcf-5f66-479f-9ae5-0e03a137cf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:57.930294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602bcf-5f66-479f-9ae5-0e03a137cf3c None admin] PATH_INFO: `/v3/users/daad4a1e651f40c4a8e915f53c3d333a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:57.931514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602bcf-5f66-479f-9ae5-0e03a137cf3c None admin] RBAC: Authorizing `identity:get_user(user_id=daad4a1e651f40c4a8e915f53c3d333a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:57.932975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602bcf-5f66-479f-9ae5-0e03a137cf3c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:57.934819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 103/205] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:57 2022] GET /identity/v3/users/daad4a1e651f40c4a8e915f53c3d333a => generated 289 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:57.940138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:58.073637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:58.081505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747342da-f4d6-447b-8f06-54ffbc94e31d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:58.130734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8450dda-0f3e-454f-ab24-d540012797c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:58.131399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450dda-0f3e-454f-ab24-d540012797c8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.132024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450dda-0f3e-454f-ab24-d540012797c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.132421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8450dda-0f3e-454f-ab24-d540012797c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.225610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450dda-0f3e-454f-ab24-d540012797c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:58.227834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8450dda-0f3e-454f-ab24-d540012797c8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:58.337778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:58.342625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 103/206] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:58.343339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e760-0f51-49df-a533-85d9556a4792 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.344235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e760-0f51-49df-a533-85d9556a4792 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.344712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4e760-0f51-49df-a533-85d9556a4792 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.346430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 104/207] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:58.350764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8450dda-0f3e-454f-ab24-d540012797c8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:58.352965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e65e4-38de-49d9-9464-6e4a3f695808 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.353509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e65e4-38de-49d9-9464-6e4a3f695808 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.353980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e65e4-38de-49d9-9464-6e4a3f695808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.371860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5c5e65e4-38de-49d9-9464-6e4a3f695808 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:58.373493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5c5e65e4-38de-49d9-9464-6e4a3f695808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:58.462571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8450dda-0f3e-454f-ab24-d540012797c8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:58.465270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 105/208] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:08:58.466437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84029a30-ce29-4845-857c-093375fa38f9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.466834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84029a30-ce29-4845-857c-093375fa38f9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.466998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84029a30-ce29-4845-857c-093375fa38f9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.468264 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 106/209] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:58.478737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b33cf-0742-40f4-8312-08c606d3cea9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.479025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b33cf-0742-40f4-8312-08c606d3cea9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.479309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b33cf-0742-40f4-8312-08c606d3cea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.506146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607b067-bbbc-4306-adcc-2c3beaddf317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:58.507335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b067-bbbc-4306-adcc-2c3beaddf317 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.507612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b067-bbbc-4306-adcc-2c3beaddf317 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.507894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607b067-bbbc-4306-adcc-2c3beaddf317 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.508547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-912b33cf-0742-40f4-8312-08c606d3cea9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:58.509113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607b067-bbbc-4306-adcc-2c3beaddf317 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:58.510207 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-912b33cf-0742-40f4-8312-08c606d3cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:58.510606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607b067-bbbc-4306-adcc-2c3beaddf317 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:58.512369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 104/210] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:58 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:58.513251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607b067-bbbc-4306-adcc-2c3beaddf317 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:58.548958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:58.579432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:58.597617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 107/211] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:58.597617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b33cf-0742-40f4-8312-08c606d3cea9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:58.642932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607b067-bbbc-4306-adcc-2c3beaddf317 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:58.683206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfa4ce-dceb-40a9-901c-d7be449ba50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:58.684415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfa4ce-dceb-40a9-901c-d7be449ba50a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.684905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfa4ce-dceb-40a9-901c-d7be449ba50a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.685406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfa4ce-dceb-40a9-901c-d7be449ba50a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.696986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bfa4ce-dceb-40a9-901c-d7be449ba50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b7bfa4ce-dceb-40a9-901c-d7be449ba50a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:58.704800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:58.705768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:58.706629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 105/212] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:58.707284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfa4ce-dceb-40a9-901c-d7be449ba50a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:58.720704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b33cf-0742-40f4-8312-08c606d3cea9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:58.756931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2a832-e76e-42be-b0e5-d1a2bcf4cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:58.757607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a832-e76e-42be-b0e5-d1a2bcf4cfba None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.757896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a832-e76e-42be-b0e5-d1a2bcf4cfba None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.758176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a832-e76e-42be-b0e5-d1a2bcf4cfba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.759023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2a832-e76e-42be-b0e5-d1a2bcf4cfba None admin] RBAC: Adding query filter params (role.id=70ff25a62a5149a98cdf09458ee3fa10, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167, user.id=daad4a1e651f40c4a8e915f53c3d333a) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:58.759338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2a832-e76e-42be-b0e5-d1a2bcf4cfba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:58.760946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2a832-e76e-42be-b0e5-d1a2bcf4cfba None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:58.773492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 108/213] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:58 2022] GET /identity/v3/role_assignments?user.id=daad4a1e651f40c4a8e915f53c3d333a&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=70ff25a62a5149a98cdf09458ee3fa10 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:58.775684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65491a35-2076-4938-a9de-3240cdf67854 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.776123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65491a35-2076-4938-a9de-3240cdf67854 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.776549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65491a35-2076-4938-a9de-3240cdf67854 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.783601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 109/214] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:58.784884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65491a35-2076-4938-a9de-3240cdf67854 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:58.813671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:58.830041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfa4ce-dceb-40a9-901c-d7be449ba50a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:58.867764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:58.879916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6c637-f282-4a3d-9e98-5663ae0f9cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:58.880533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c637-f282-4a3d-9e98-5663ae0f9cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.880958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c637-f282-4a3d-9e98-5663ae0f9cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.881385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c637-f282-4a3d-9e98-5663ae0f9cd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.883141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c637-f282-4a3d-9e98-5663ae0f9cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:58.885531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c637-f282-4a3d-9e98-5663ae0f9cd9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:58.886755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 106/215] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:08:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:58.893100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128409f-f2c4-4d71-9cab-b266e66fca23 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.893450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128409f-f2c4-4d71-9cab-b266e66fca23 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.893877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f128409f-f2c4-4d71-9cab-b266e66fca23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.912529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f128409f-f2c4-4d71-9cab-b266e66fca23 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:58.914253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f128409f-f2c4-4d71-9cab-b266e66fca23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:58.928448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65491a35-2076-4938-a9de-3240cdf67854 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:58.975634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:58.976056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:58.976344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:58.976620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:58.977429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:58.977742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:58.979151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:58.993346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 110/216] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:08:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:58.993346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.013199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 107/217] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:59.013199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128409f-f2c4-4d71-9cab-b266e66fca23 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.098475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:59.122270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd168ca3-f7f3-4322-8e98-f6a12a309606 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:59.129163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f128409f-f2c4-4d71-9cab-b266e66fca23 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:59.154250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:59.160696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9ac70-ba74-4176-b16b-92792296011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:59.161405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ac70-ba74-4176-b16b-92792296011d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.161693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ac70-ba74-4176-b16b-92792296011d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.161967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd9ac70-ba74-4176-b16b-92792296011d None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.162683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9ac70-ba74-4176-b16b-92792296011d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:59.164180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd9ac70-ba74-4176-b16b-92792296011d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:59.167057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210607ad-c5c3-4de9-b055-aed4dba7aee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:59.168128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210607ad-c5c3-4de9-b055-aed4dba7aee5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.168573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210607ad-c5c3-4de9-b055-aed4dba7aee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.169028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210607ad-c5c3-4de9-b055-aed4dba7aee5 None admin] PATH_INFO: `/v3/users/b40a4b06fb05405db1aeb2ef76daad03` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.185680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 111/218] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:08:58 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:08:59.185861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210607ad-c5c3-4de9-b055-aed4dba7aee5 None admin] RBAC: Authorizing `identity:get_user(user_id=b40a4b06fb05405db1aeb2ef76daad03)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:59.186486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9ac70-ba74-4176-b16b-92792296011d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.187274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210607ad-c5c3-4de9-b055-aed4dba7aee5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:59.189016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 108/219] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/users/b40a4b06fb05405db1aeb2ef76daad03 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:59.190329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ea02-088a-4d4c-9d58-a1a90cffd4e7 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.190617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ea02-088a-4d4c-9d58-a1a90cffd4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.190894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ea02-088a-4d4c-9d58-a1a90cffd4e7 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.192372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 109/220] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:08:59.194645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564ea02-088a-4d4c-9d58-a1a90cffd4e7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.305265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd9ac70-ba74-4176-b16b-92792296011d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:59.312209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564ea02-088a-4d4c-9d58-a1a90cffd4e7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:59.341988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6368094-1cbb-4146-9fc5-9edab889136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:59.342845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6368094-1cbb-4146-9fc5-9edab889136c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.343102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6368094-1cbb-4146-9fc5-9edab889136c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.343389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6368094-1cbb-4146-9fc5-9edab889136c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.346898 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:59.351020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b014947-4f6f-41db-9ba7-3606bfca9085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:59.351694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014947-4f6f-41db-9ba7-3606bfca9085 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.351955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014947-4f6f-41db-9ba7-3606bfca9085 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.352252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b014947-4f6f-41db-9ba7-3606bfca9085 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.358733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6368094-1cbb-4146-9fc5-9edab889136c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b6368094-1cbb-4146-9fc5-9edab889136c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:59.366557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:59.367515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:08:59.368307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 112/221] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:59.368976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeefd5-763b-432d-a801-4703287970ea None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.369280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeefd5-763b-432d-a801-4703287970ea None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.369578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeeefd5-763b-432d-a801-4703287970ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.394032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-faeeefd5-763b-432d-a801-4703287970ea None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:08:59.395148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-faeeefd5-763b-432d-a801-4703287970ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:08:59.417695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:59.440408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b014947-4f6f-41db-9ba7-3606bfca9085 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:59.441998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b014947-4f6f-41db-9ba7-3606bfca9085 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:59.490582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 113/222] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:08:59.496428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeeefd5-763b-432d-a801-4703287970ea None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.562982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 110/223] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:08:59.568278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b014947-4f6f-41db-9ba7-3606bfca9085 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.610867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeeefd5-763b-432d-a801-4703287970ea None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:59.638944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:59.656005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:59.656607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.656889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.657162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.657947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:08:59.658256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:59.659659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:59.671650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 114/224] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:59.677070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.686228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:59.706547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b014947-4f6f-41db-9ba7-3606bfca9085 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:59.744625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b7fe-cb39-489f-af28-a4fc4520c6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:59.745403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b7fe-cb39-489f-af28-a4fc4520c6bf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.745689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b7fe-cb39-489f-af28-a4fc4520c6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.745966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528b7fe-cb39-489f-af28-a4fc4520c6bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.757142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528b7fe-cb39-489f-af28-a4fc4520c6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8528b7fe-cb39-489f-af28-a4fc4520c6bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:08:59.765416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:08:59.766347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:08:59.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 111/225] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:08:59.767825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b7fe-cb39-489f-af28-a4fc4520c6bf None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.805268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4bf60-3fd6-4d5f-9a06-8598815f0598 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:59.849484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe869ef-6abf-4b5a-95c8-c3005004ad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:59.850774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe869ef-6abf-4b5a-95c8-c3005004ad6b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.851251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe869ef-6abf-4b5a-95c8-c3005004ad6b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.851722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe869ef-6abf-4b5a-95c8-c3005004ad6b None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.853753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe869ef-6abf-4b5a-95c8-c3005004ad6b None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:59.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe869ef-6abf-4b5a-95c8-c3005004ad6b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:59.858850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 115/226] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:59.860305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe869ef-6abf-4b5a-95c8-c3005004ad6b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.885606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528b7fe-cb39-489f-af28-a4fc4520c6bf None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:08:59.927630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:59.930950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a82557-e82e-4851-8aac-1f4d27d1254a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:08:59.933134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a82557-e82e-4851-8aac-1f4d27d1254a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:08:59.933465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a82557-e82e-4851-8aac-1f4d27d1254a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:08:59.933750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a82557-e82e-4851-8aac-1f4d27d1254a None admin] PATH_INFO: `/v3/users/5df3f5a3c0d14010a8317fa68a23c2db` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:08:59.935614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a82557-e82e-4851-8aac-1f4d27d1254a None admin] RBAC: Authorizing `identity:get_user(user_id=5df3f5a3c0d14010a8317fa68a23c2db)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:08:59.937405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a82557-e82e-4851-8aac-1f4d27d1254a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:08:59.939654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 112/227] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/users/5df3f5a3c0d14010a8317fa68a23c2db => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:08:59.941047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a82557-e82e-4851-8aac-1f4d27d1254a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:08:59.957002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:08:59.979947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe869ef-6abf-4b5a-95c8-c3005004ad6b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:00.016900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:00.017866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:00.018250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:00.018679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:00.019622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:00.020418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:00.021960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:00.031926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 116/228] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:00.032888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:00.066198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a82557-e82e-4851-8aac-1f4d27d1254a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:00.108227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:00.108888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:00.109169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:00.109518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:00.113690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] RBAC: Adding query filter params (role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=fc31ecc9189941fa909edb765d79c496, user.id=b40a4b06fb05405db1aeb2ef76daad03) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:00.114286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:00.117634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:00.125944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 113/229] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:08:59 2022] GET /identity/v3/role_assignments?user.id=b40a4b06fb05405db1aeb2ef76daad03&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:00.127410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:00.201047 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf17c4-886f-46b6-8dbc-c8e660b7cc25 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:00.245668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:00.274020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:00.277465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e11c97-6972-45a8-b8b4-15fa00b86de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:00.278634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e11c97-6972-45a8-b8b4-15fa00b86de6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:00.280403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e11c97-6972-45a8-b8b4-15fa00b86de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:00.280931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e11c97-6972-45a8-b8b4-15fa00b86de6 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:00.282545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e11c97-6972-45a8-b8b4-15fa00b86de6 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:00.284256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e11c97-6972-45a8-b8b4-15fa00b86de6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:00.286304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 117/230] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:00 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:00.292384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e11c97-6972-45a8-b8b4-15fa00b86de6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:00.313331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ecb6d-cbf5-4468-aa71-401a54ad603a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:00.349846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee20e2f-4dd8-457f-ba7d-69675a2b5da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:00.351117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20e2f-4dd8-457f-ba7d-69675a2b5da2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:00.351117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20e2f-4dd8-457f-ba7d-69675a2b5da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:00.351396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee20e2f-4dd8-457f-ba7d-69675a2b5da2 None admin] PATH_INFO: `/v3/users/5df3f5a3c0d14010a8317fa68a23c2db` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:00.352663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee20e2f-4dd8-457f-ba7d-69675a2b5da2 None admin] RBAC: Authorizing `identity:get_user(user_id=5df3f5a3c0d14010a8317fa68a23c2db)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:00.357224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee20e2f-4dd8-457f-ba7d-69675a2b5da2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:00.357224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 114/231] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:00 2022] GET /identity/v3/users/5df3f5a3c0d14010a8317fa68a23c2db => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:00.363997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee20e2f-4dd8-457f-ba7d-69675a2b5da2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:00.530832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e11c97-6972-45a8-b8b4-15fa00b86de6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:00.555138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee20e2f-4dd8-457f-ba7d-69675a2b5da2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:00.567521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdb42d-fde8-4bc2-8dcb-f78b88494fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:00.568448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb42d-fde8-4bc2-8dcb-f78b88494fbd None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:00.568738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb42d-fde8-4bc2-8dcb-f78b88494fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:00.569019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fdb42d-fde8-4bc2-8dcb-f78b88494fbd None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496/users/5df3f5a3c0d14010a8317fa68a23c2db/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:00.571935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdb42d-fde8-4bc2-8dcb-f78b88494fbd None admin] RBAC: Authorizing `identity:create_grant(project_id=fc31ecc9189941fa909edb765d79c496, user_id=5df3f5a3c0d14010a8317fa68a23c2db, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:00.573539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fdb42d-fde8-4bc2-8dcb-f78b88494fbd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:00.595935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 118/232] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:00 2022] PUT /identity/v3/projects/fc31ecc9189941fa909edb765d79c496/users/5df3f5a3c0d14010a8317fa68a23c2db/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:00.624072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6399b-eba6-4f19-aa0b-0f5de99d70d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:00.625015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6399b-eba6-4f19-aa0b-0f5de99d70d5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:00.625319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6399b-eba6-4f19-aa0b-0f5de99d70d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:00.625596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6399b-eba6-4f19-aa0b-0f5de99d70d5 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:00.627337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6399b-eba6-4f19-aa0b-0f5de99d70d5 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:00.628875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6399b-eba6-4f19-aa0b-0f5de99d70d5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:00.631033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 115/233] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:00 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:00.637695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdb42d-fde8-4bc2-8dcb-f78b88494fbd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:00.726660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:00.805712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fdb42d-fde8-4bc2-8dcb-f78b88494fbd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:00.844209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944abf4c-5c0f-4ff2-b4ec-9a19234e0545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:00.845134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944abf4c-5c0f-4ff2-b4ec-9a19234e0545 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:00.845449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944abf4c-5c0f-4ff2-b4ec-9a19234e0545 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:00.845734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944abf4c-5c0f-4ff2-b4ec-9a19234e0545 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496/users/5df3f5a3c0d14010a8317fa68a23c2db/roles/774686ef0c4c42ba97f56e75b67d313e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:00.848053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944abf4c-5c0f-4ff2-b4ec-9a19234e0545 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc31ecc9189941fa909edb765d79c496, user_id=5df3f5a3c0d14010a8317fa68a23c2db, role_id=774686ef0c4c42ba97f56e75b67d313e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:00.849563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944abf4c-5c0f-4ff2-b4ec-9a19234e0545 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:00.867551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 119/234] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:00 2022] PUT /identity/v3/projects/fc31ecc9189941fa909edb765d79c496/users/5df3f5a3c0d14010a8317fa68a23c2db/roles/774686ef0c4c42ba97f56e75b67d313e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:01.655671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68fdd-0eae-46de-9109-6c7298665095 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:01.656184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68fdd-0eae-46de-9109-6c7298665095 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:01.656258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc68fdd-0eae-46de-9109-6c7298665095 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:01.657925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 116/235] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:01.663117 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a9ff7-34cd-4c76-8072-24d2d24f6f84 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:01.663535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a9ff7-34cd-4c76-8072-24d2d24f6f84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:01.663763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a9ff7-34cd-4c76-8072-24d2d24f6f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:01.690200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6d0a9ff7-34cd-4c76-8072-24d2d24f6f84 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:01.691389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d0a9ff7-34cd-4c76-8072-24d2d24f6f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:01.781753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 120/236] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:01.817516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc68fdd-0eae-46de-9109-6c7298665095 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:01.838147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ac4e-2d56-4666-8bc0-a4405cd6f17f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:01.838451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ac4e-2d56-4666-8bc0-a4405cd6f17f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:01.838704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00ac4e-2d56-4666-8bc0-a4405cd6f17f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:01.840086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 121/237] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:01.841351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:01.847889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327026a6-5b71-4544-ac20-7c7fa49d03aa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:01.848456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327026a6-5b71-4544-ac20-7c7fa49d03aa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:01.848897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327026a6-5b71-4544-ac20-7c7fa49d03aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:01.867933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-327026a6-5b71-4544-ac20-7c7fa49d03aa None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:01.869523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-327026a6-5b71-4544-ac20-7c7fa49d03aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:01.913515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:01.994431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 122/238] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:02.003120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc68fdd-0eae-46de-9109-6c7298665095 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:02.031139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327026a6-5b71-4544-ac20-7c7fa49d03aa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:02.058409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525689f-d233-4b42-a539-b1d856bde35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:02.059712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525689f-d233-4b42-a539-b1d856bde35d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:02.061485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525689f-d233-4b42-a539-b1d856bde35d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:02.061914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525689f-d233-4b42-a539-b1d856bde35d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:02.087683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525689f-d233-4b42-a539-b1d856bde35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5525689f-d233-4b42-a539-b1d856bde35d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:02.095678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:02.096732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:02.097772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:02.110804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 117/239] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:02.127030 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525689f-d233-4b42-a539-b1d856bde35d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:02.197774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327026a6-5b71-4544-ac20-7c7fa49d03aa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:02.236547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313f984-9057-4b00-a972-a5e6f855f8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:02.237694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313f984-9057-4b00-a972-a5e6f855f8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:02.238119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313f984-9057-4b00-a972-a5e6f855f8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:02.238537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313f984-9057-4b00-a972-a5e6f855f8c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:02.240030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313f984-9057-4b00-a972-a5e6f855f8c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:02.242471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313f984-9057-4b00-a972-a5e6f855f8c3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:02.244318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 123/240] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:02.250099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313f984-9057-4b00-a972-a5e6f855f8c3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:02.293582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:02.319979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525689f-d233-4b42-a539-b1d856bde35d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:02.327609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:02.369145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:02.370123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:02.370548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:02.370964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:02.371862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:02.372321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:02.373863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:02.386322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 118/241] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:02.394021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:02.448355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313f984-9057-4b00-a972-a5e6f855f8c3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:02.523378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc697bf-0fd3-48b6-879c-430b5c981f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:02.524325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc697bf-0fd3-48b6-879c-430b5c981f6c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:02.524749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc697bf-0fd3-48b6-879c-430b5c981f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:02.525163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc697bf-0fd3-48b6-879c-430b5c981f6c None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:02.526098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc697bf-0fd3-48b6-879c-430b5c981f6c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:02.527897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc697bf-0fd3-48b6-879c-430b5c981f6c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:02.571467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 124/242] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:09:02 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:09:02.592970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4dc9c-03d2-4c6b-b988-1975a227e278 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:02.635103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:02.640590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77805fe-6501-4b5e-9577-2d00e7213b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:02.641620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77805fe-6501-4b5e-9577-2d00e7213b0c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:02.641932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77805fe-6501-4b5e-9577-2d00e7213b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:02.642211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77805fe-6501-4b5e-9577-2d00e7213b0c None admin] PATH_INFO: `/v3/users/5ea00c3789304f3f826adeff3f959dad` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:02.643577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c94c2-b7c5-4b72-818e-35ae35c731b0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:02.643867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c94c2-b7c5-4b72-818e-35ae35c731b0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:02.644150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c94c2-b7c5-4b72-818e-35ae35c731b0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:02.645579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 125/243] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:02.651245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3bef8-d45e-49a9-84f9-f21c68aaa250 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:02.651528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3bef8-d45e-49a9-84f9-f21c68aaa250 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:02.651954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3bef8-d45e-49a9-84f9-f21c68aaa250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:02.658086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77805fe-6501-4b5e-9577-2d00e7213b0c None admin] RBAC: Authorizing `identity:get_user(user_id=5ea00c3789304f3f826adeff3f959dad)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:02.660435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77805fe-6501-4b5e-9577-2d00e7213b0c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:02.662565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 119/244] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:02 2022] GET /identity/v3/users/5ea00c3789304f3f826adeff3f959dad => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:02.670281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77805fe-6501-4b5e-9577-2d00e7213b0c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:02.694479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f5b3bef8-d45e-49a9-84f9-f21c68aaa250 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:02.695739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5b3bef8-d45e-49a9-84f9-f21c68aaa250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:02.783622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77805fe-6501-4b5e-9577-2d00e7213b0c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:02.826287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da28c8d-ff29-463e-8d1a-d25c00d66f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:02.827181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da28c8d-ff29-463e-8d1a-d25c00d66f44 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:02.827689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da28c8d-ff29-463e-8d1a-d25c00d66f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:02.828114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da28c8d-ff29-463e-8d1a-d25c00d66f44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:02.830101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 126/245] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:02.852574 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3bef8-d45e-49a9-84f9-f21c68aaa250 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:02.883664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:02.895759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:02.907569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da28c8d-ff29-463e-8d1a-d25c00d66f44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:02.909142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da28c8d-ff29-463e-8d1a-d25c00d66f44 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:02.976239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3bef8-d45e-49a9-84f9-f21c68aaa250 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:03.016627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71494632-26e3-4dbd-81ca-72d3008bc7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:03.017745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71494632-26e3-4dbd-81ca-72d3008bc7ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.018172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71494632-26e3-4dbd-81ca-72d3008bc7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.018714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71494632-26e3-4dbd-81ca-72d3008bc7ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.019422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 120/246] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:03.025638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da28c8d-ff29-463e-8d1a-d25c00d66f44 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:03.049390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71494632-26e3-4dbd-81ca-72d3008bc7ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-71494632-26e3-4dbd-81ca-72d3008bc7ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:03.057833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:03.058915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:03.059941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:03.062325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 127/247] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:03.075125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71494632-26e3-4dbd-81ca-72d3008bc7ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:03.145644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:03.161428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da28c8d-ff29-463e-8d1a-d25c00d66f44 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:03.181256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:03.207108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71494632-26e3-4dbd-81ca-72d3008bc7ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:03.213673 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125e7c4-b47a-46bc-b424-05f62bd9dcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:03.214558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125e7c4-b47a-46bc-b424-05f62bd9dcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.214871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125e7c4-b47a-46bc-b424-05f62bd9dcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.215224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125e7c4-b47a-46bc-b424-05f62bd9dcb7 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.216695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125e7c4-b47a-46bc-b424-05f62bd9dcb7 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:03.218464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125e7c4-b47a-46bc-b424-05f62bd9dcb7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:03.220928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 121/248] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:03 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:03.226777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125e7c4-b47a-46bc-b424-05f62bd9dcb7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:03.259541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:03.260423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.260845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.261325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.262549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:03.263409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:03.265420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:03.277755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 128/249] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:03.283435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:03.357143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125e7c4-b47a-46bc-b424-05f62bd9dcb7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:03.430037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3306eab-3100-4147-b080-3ed0e3205cc8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:03.432621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff55cf-da2e-4d7b-9175-5aac332fbb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:03.433464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff55cf-da2e-4d7b-9175-5aac332fbb90 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.433832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff55cf-da2e-4d7b-9175-5aac332fbb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.434279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff55cf-da2e-4d7b-9175-5aac332fbb90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.435878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff55cf-da2e-4d7b-9175-5aac332fbb90 None admin] RBAC: Adding query filter params (role.id=70ff25a62a5149a98cdf09458ee3fa10, scope.project.id=fc31ecc9189941fa909edb765d79c496, user.id=5ea00c3789304f3f826adeff3f959dad) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:03.436638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff55cf-da2e-4d7b-9175-5aac332fbb90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:03.439394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff55cf-da2e-4d7b-9175-5aac332fbb90 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:03.455690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 122/250] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:03 2022] GET /identity/v3/role_assignments?user.id=5ea00c3789304f3f826adeff3f959dad&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=70ff25a62a5149a98cdf09458ee3fa10 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:03.457769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74b92-2c88-4648-9cf5-6b073deabaa3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.458247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74b92-2c88-4648-9cf5-6b073deabaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.458726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a74b92-2c88-4648-9cf5-6b073deabaa3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.460782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 123/251] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:03.466879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9952f-1fc5-4fe8-964b-3a0ae15f6be0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.467586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9952f-1fc5-4fe8-964b-3a0ae15f6be0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.468064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9952f-1fc5-4fe8-964b-3a0ae15f6be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.482325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58354c7-ed89-428a-97a9-a82d44b6b553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:03.483237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58354c7-ed89-428a-97a9-a82d44b6b553 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.483524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58354c7-ed89-428a-97a9-a82d44b6b553 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.483798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58354c7-ed89-428a-97a9-a82d44b6b553 None admin] PATH_INFO: `/v3/users/b40a4b06fb05405db1aeb2ef76daad03` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.485112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58354c7-ed89-428a-97a9-a82d44b6b553 None admin] RBAC: Authorizing `identity:get_user(user_id=b40a4b06fb05405db1aeb2ef76daad03)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:03.495152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4ed9952f-1fc5-4fe8-964b-3a0ae15f6be0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:03.495874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58354c7-ed89-428a-97a9-a82d44b6b553 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:03.496824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4ed9952f-1fc5-4fe8-964b-3a0ae15f6be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:03.497903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 129/252] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:03 2022] GET /identity/v3/users/b40a4b06fb05405db1aeb2ef76daad03 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:03.503332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58354c7-ed89-428a-97a9-a82d44b6b553 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:03.517868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:03.525473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:03.582643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 124/253] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:03.594841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171576-aec3-4cdb-94ba-09af725310c2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.595514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171576-aec3-4cdb-94ba-09af725310c2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.595970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f171576-aec3-4cdb-94ba-09af725310c2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.597991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 125/254] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:03.604404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98075f55-3611-4641-b037-0d73925e0aa4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.604968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98075f55-3611-4641-b037-0d73925e0aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.605469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98075f55-3611-4641-b037-0d73925e0aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.642373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-98075f55-3611-4641-b037-0d73925e0aa4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:03.643905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-98075f55-3611-4641-b037-0d73925e0aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:03.659732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58354c7-ed89-428a-97a9-a82d44b6b553 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:03.703146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fdb3a-e3b4-44e5-a5bb-80994afb634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:03.705686 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fdb3a-e3b4-44e5-a5bb-80994afb634d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.706215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fdb3a-e3b4-44e5-a5bb-80994afb634d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.706728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4fdb3a-e3b4-44e5-a5bb-80994afb634d None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.708423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fdb3a-e3b4-44e5-a5bb-80994afb634d None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:03.710859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4fdb3a-e3b4-44e5-a5bb-80994afb634d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:03.714193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 130/255] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:03 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:03.715405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fdb3a-e3b4-44e5-a5bb-80994afb634d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:03.777082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 126/256] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:03.778000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98075f55-3611-4641-b037-0d73925e0aa4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:03.855738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:03.889858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4fdb3a-e3b4-44e5-a5bb-80994afb634d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:03.896783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:03.922169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98075f55-3611-4641-b037-0d73925e0aa4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:03.933323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3984450-8d23-4bc7-b304-fe570e6785ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:03.934294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3984450-8d23-4bc7-b304-fe570e6785ff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.934689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3984450-8d23-4bc7-b304-fe570e6785ff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.935063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3984450-8d23-4bc7-b304-fe570e6785ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.947607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3984450-8d23-4bc7-b304-fe570e6785ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c3984450-8d23-4bc7-b304-fe570e6785ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:03.956053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:03.957189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:03.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:03.961100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 131/257] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:09:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:03.962022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3984450-8d23-4bc7-b304-fe570e6785ff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:03.970488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821dea2f-67f5-406d-b87f-63e4aa2ee087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:03.971630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dea2f-67f5-406d-b87f-63e4aa2ee087 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:03.972066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dea2f-67f5-406d-b87f-63e4aa2ee087 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:03.972483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821dea2f-67f5-406d-b87f-63e4aa2ee087 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496/users/b40a4b06fb05405db1aeb2ef76daad03/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:03.975223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821dea2f-67f5-406d-b87f-63e4aa2ee087 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc31ecc9189941fa909edb765d79c496, user_id=b40a4b06fb05405db1aeb2ef76daad03, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:03.977260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821dea2f-67f5-406d-b87f-63e4aa2ee087 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:04.002487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 127/258] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:03 2022] PUT /identity/v3/projects/fc31ecc9189941fa909edb765d79c496/users/b40a4b06fb05405db1aeb2ef76daad03/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:04.003616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821dea2f-67f5-406d-b87f-63e4aa2ee087 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:04.168745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3984450-8d23-4bc7-b304-fe570e6785ff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:04.253592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32acbf2-8921-4d45-b0d4-feecaf4a92b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:04.255148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acbf2-8921-4d45-b0d4-feecaf4a92b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:04.255760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acbf2-8921-4d45-b0d4-feecaf4a92b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:04.256216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acbf2-8921-4d45-b0d4-feecaf4a92b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:04.265911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:04.273768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32acbf2-8921-4d45-b0d4-feecaf4a92b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a32acbf2-8921-4d45-b0d4-feecaf4a92b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:04.283469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:04.284427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:04.285284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:04.288159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 132/259] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:09:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:04.294459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32acbf2-8921-4d45-b0d4-feecaf4a92b6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:04.314610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821dea2f-67f5-406d-b87f-63e4aa2ee087 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:04.356697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:04.357722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:04.358193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:04.358643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:04.359586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:04.360179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:04.361790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:04.371960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 128/260] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:04.377720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:04.382940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:04.492051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab187e-eb3b-408c-9123-be12fb6ad512 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:04.563235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac06a0-f602-4d86-9089-fd9e5ced0e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:04.564463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac06a0-f602-4d86-9089-fd9e5ced0e56 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:04.564925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac06a0-f602-4d86-9089-fd9e5ced0e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:04.565416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ac06a0-f602-4d86-9089-fd9e5ced0e56 None admin] PATH_INFO: `/v3/users/5df3f5a3c0d14010a8317fa68a23c2db` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:04.567216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac06a0-f602-4d86-9089-fd9e5ced0e56 None admin] RBAC: Authorizing `identity:get_user(user_id=5df3f5a3c0d14010a8317fa68a23c2db)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:04.571485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ac06a0-f602-4d86-9089-fd9e5ced0e56 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:04.573622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 129/261] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:04 2022] GET /identity/v3/users/5df3f5a3c0d14010a8317fa68a23c2db => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:04.579618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac06a0-f602-4d86-9089-fd9e5ced0e56 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:04.588672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32acbf2-8921-4d45-b0d4-feecaf4a92b6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:04.682461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:04.683268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:04.683403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:04.683663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:04.683946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:04.684712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:04.685020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:04.686470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:04.707180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 133/262] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:04.722248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:04.736814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ac06a0-f602-4d86-9089-fd9e5ced0e56 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:04.805947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89223310-9133-42ff-a91e-4f466cb80492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:04.806636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89223310-9133-42ff-a91e-4f466cb80492 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:04.806927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89223310-9133-42ff-a91e-4f466cb80492 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:04.807211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89223310-9133-42ff-a91e-4f466cb80492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:04.857463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:04.903161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f094bc-a1d0-4b62-90c5-a46bf7d37ca5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:04.943801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac1e2e-d364-4fc1-ae5e-913d43ac2dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:04.945246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac1e2e-d364-4fc1-ae5e-913d43ac2dfd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:04.945735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac1e2e-d364-4fc1-ae5e-913d43ac2dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:04.947529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac1e2e-d364-4fc1-ae5e-913d43ac2dfd None admin] PATH_INFO: `/v3/users/5df3f5a3c0d14010a8317fa68a23c2db` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:04.949035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac1e2e-d364-4fc1-ae5e-913d43ac2dfd None admin] RBAC: Authorizing `identity:get_user(user_id=5df3f5a3c0d14010a8317fa68a23c2db)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:04.952273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac1e2e-d364-4fc1-ae5e-913d43ac2dfd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:04.955461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 134/263] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:04 2022] GET /identity/v3/users/5df3f5a3c0d14010a8317fa68a23c2db => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:04.962503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac1e2e-d364-4fc1-ae5e-913d43ac2dfd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:04.986978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89223310-9133-42ff-a91e-4f466cb80492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:04.991579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89223310-9133-42ff-a91e-4f466cb80492 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:05.128638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac1e2e-d364-4fc1-ae5e-913d43ac2dfd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:05.156780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:05.157693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:05.167357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020b763-78d2-40f4-ad43-7fd89c4a60eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:05.168358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020b763-78d2-40f4-ad43-7fd89c4a60eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:05.168853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020b763-78d2-40f4-ad43-7fd89c4a60eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:05.169486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020b763-78d2-40f4-ad43-7fd89c4a60eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:05.233023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 130/264] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:09:05.238877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89223310-9133-42ff-a91e-4f466cb80492 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:05.244896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020b763-78d2-40f4-ad43-7fd89c4a60eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:05.246745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020b763-78d2-40f4-ad43-7fd89c4a60eb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:05.368242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 135/265] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:05.376715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020b763-78d2-40f4-ad43-7fd89c4a60eb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:05.405515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89223310-9133-42ff-a91e-4f466cb80492 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:05.438702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:05.449783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03f959-0220-497c-a071-86a23e092b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:05.450689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f959-0220-497c-a071-86a23e092b43 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:05.450971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f959-0220-497c-a071-86a23e092b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:05.451246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03f959-0220-497c-a071-86a23e092b43 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:05.452485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03f959-0220-497c-a071-86a23e092b43 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:05.454272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03f959-0220-497c-a071-86a23e092b43 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:05.457329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 131/266] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:05 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:05.463542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03f959-0220-497c-a071-86a23e092b43 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:05.542263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:05.593570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020b763-78d2-40f4-ad43-7fd89c4a60eb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:05.659089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03f959-0220-497c-a071-86a23e092b43 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:05.661144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63717-0620-4c0a-94dd-471c422cbc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:05.662088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63717-0620-4c0a-94dd-471c422cbc49 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:05.662372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63717-0620-4c0a-94dd-471c422cbc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:05.662644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c63717-0620-4c0a-94dd-471c422cbc49 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:05.666465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c63717-0620-4c0a-94dd-471c422cbc49 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:05.667969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c63717-0620-4c0a-94dd-471c422cbc49 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:05.670829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 136/267] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:05 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:05.679425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63717-0620-4c0a-94dd-471c422cbc49 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:05.740531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a8932-9dfd-4eb4-b39f-2667b7e35bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:05.741271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8932-9dfd-4eb4-b39f-2667b7e35bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:05.741559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8932-9dfd-4eb4-b39f-2667b7e35bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:05.742009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a8932-9dfd-4eb4-b39f-2667b7e35bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:05.742865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a8932-9dfd-4eb4-b39f-2667b7e35bf8 None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=fc31ecc9189941fa909edb765d79c496, user.id=5df3f5a3c0d14010a8317fa68a23c2db) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:05.743181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a8932-9dfd-4eb4-b39f-2667b7e35bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:05.744652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a8932-9dfd-4eb4-b39f-2667b7e35bf8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:05.765164 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 132/268] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:05 2022] GET /identity/v3/role_assignments?user.id=5df3f5a3c0d14010a8317fa68a23c2db&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:05.856183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:05.862797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c63717-0620-4c0a-94dd-471c422cbc49 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:05.907472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ac1fe-6c21-4590-8ebe-8ca858debb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:05.908139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ac1fe-6c21-4590-8ebe-8ca858debb95 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:05.908419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ac1fe-6c21-4590-8ebe-8ca858debb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:05.908688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ac1fe-6c21-4590-8ebe-8ca858debb95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:05.909610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ac1fe-6c21-4590-8ebe-8ca858debb95 None admin] RBAC: Adding query filter params (role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=fc31ecc9189941fa909edb765d79c496, user.id=5df3f5a3c0d14010a8317fa68a23c2db) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:05.909862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ac1fe-6c21-4590-8ebe-8ca858debb95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:05.911327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ac1fe-6c21-4590-8ebe-8ca858debb95 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:05.922365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 137/269] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:05 2022] GET /identity/v3/role_assignments?user.id=5df3f5a3c0d14010a8317fa68a23c2db&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:06.235735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe9049-6765-422d-86ae-eebef6125ef0 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.236782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe9049-6765-422d-86ae-eebef6125ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.237286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe9049-6765-422d-86ae-eebef6125ef0 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.239092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 133/270] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:06.254705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e9310-debc-452a-a5a4-b52de0597ca6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.255318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e9310-debc-452a-a5a4-b52de0597ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.255793 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e9310-debc-452a-a5a4-b52de0597ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.263634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec4a1-cd77-40f8-b6fd-9f6c154d8ea2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.264384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec4a1-cd77-40f8-b6fd-9f6c154d8ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.264856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ec4a1-cd77-40f8-b6fd-9f6c154d8ea2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.266748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 134/271] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:06.272586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c4de-db3b-4e15-8934-c735e2f89ee7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.273366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c4de-db3b-4e15-8934-c735e2f89ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.273843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088c4de-db3b-4e15-8934-c735e2f89ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.285746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e27e9310-debc-452a-a5a4-b52de0597ca6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:06.287259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e27e9310-debc-452a-a5a4-b52de0597ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:06.295710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5088c4de-db3b-4e15-8934-c735e2f89ee7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:06.296918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5088c4de-db3b-4e15-8934-c735e2f89ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:06.380545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 138/272] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:06.410394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e9310-debc-452a-a5a4-b52de0597ca6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:06.414078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 135/273] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:06.442277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088c4de-db3b-4e15-8934-c735e2f89ee7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:06.505510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:06.511435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:06.523243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e9310-debc-452a-a5a4-b52de0597ca6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:06.560160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0046f-8697-4cb7-bb15-bdb20c9b92d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:06.560971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0046f-8697-4cb7-bb15-bdb20c9b92d7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.561280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0046f-8697-4cb7-bb15-bdb20c9b92d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.561560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0046f-8697-4cb7-bb15-bdb20c9b92d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.573231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0046f-8697-4cb7-bb15-bdb20c9b92d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-45d0046f-8697-4cb7-bb15-bdb20c9b92d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:06.580953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:06.581907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:06.582736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 139/274] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:06.588286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0046f-8697-4cb7-bb15-bdb20c9b92d7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:06.594701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088c4de-db3b-4e15-8934-c735e2f89ee7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:06.640776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5daf0-b31b-4238-ab17-d8072f3b1cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:06.642042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5daf0-b31b-4238-ab17-d8072f3b1cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.643264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5daf0-b31b-4238-ab17-d8072f3b1cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.643762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5daf0-b31b-4238-ab17-d8072f3b1cb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.645898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5daf0-b31b-4238-ab17-d8072f3b1cb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:06.648482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5daf0-b31b-4238-ab17-d8072f3b1cb2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:06.651749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 136/275] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:06.657059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5daf0-b31b-4238-ab17-d8072f3b1cb2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:06.721980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0046f-8697-4cb7-bb15-bdb20c9b92d7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:06.757866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55bfba0-0f4a-4c5d-b43e-914429397637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:06.758461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bfba0-0f4a-4c5d-b43e-914429397637 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.758742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bfba0-0f4a-4c5d-b43e-914429397637 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.759007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55bfba0-0f4a-4c5d-b43e-914429397637 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.759766 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bfba0-0f4a-4c5d-b43e-914429397637 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:06.760074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bfba0-0f4a-4c5d-b43e-914429397637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:06.761499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55bfba0-0f4a-4c5d-b43e-914429397637 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:06.770731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 140/276] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:06.771834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf212-2626-4c19-9a9e-fd3af946e758 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.772346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf212-2626-4c19-9a9e-fd3af946e758 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.772570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bcf212-2626-4c19-9a9e-fd3af946e758 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.773894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 141/277] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:06.775827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5daf0-b31b-4238-ab17-d8072f3b1cb2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:06.777719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:06.779036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcf212-2626-4c19-9a9e-fd3af946e758 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:06.794901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:06.813023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac60ef0-ce17-4eb9-8279-d61170ff4b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:06.813994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60ef0-ce17-4eb9-8279-d61170ff4b38 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.814340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60ef0-ce17-4eb9-8279-d61170ff4b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.814618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac60ef0-ce17-4eb9-8279-d61170ff4b38 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.815324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac60ef0-ce17-4eb9-8279-d61170ff4b38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:06.816758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac60ef0-ce17-4eb9-8279-d61170ff4b38 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:06.845587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 137/278] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:09:06 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:09:06.846841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8f87c-688d-41ee-85cc-0887be457a61 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.847133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8f87c-688d-41ee-85cc-0887be457a61 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.847407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8f87c-688d-41ee-85cc-0887be457a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.872940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-45a8f87c-688d-41ee-85cc-0887be457a61 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:06.875054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-45a8f87c-688d-41ee-85cc-0887be457a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:06.900267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bcf212-2626-4c19-9a9e-fd3af946e758 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:06.939603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d519f-26e2-4c94-a411-847e1f27e25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:06.940655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d519f-26e2-4c94-a411-847e1f27e25b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:06.941012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d519f-26e2-4c94-a411-847e1f27e25b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:06.941397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d519f-26e2-4c94-a411-847e1f27e25b None admin] PATH_INFO: `/v3/users/7b03f4d117e84d1db43b54989c02efd9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:06.957591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d519f-26e2-4c94-a411-847e1f27e25b None admin] RBAC: Authorizing `identity:get_user(user_id=7b03f4d117e84d1db43b54989c02efd9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:06.959213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d519f-26e2-4c94-a411-847e1f27e25b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:06.966198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 142/279] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:06 2022] GET /identity/v3/users/7b03f4d117e84d1db43b54989c02efd9 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:06.967707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d519f-26e2-4c94-a411-847e1f27e25b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:06.976786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 138/280] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:07.000606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8f87c-688d-41ee-85cc-0887be457a61 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:07.057325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:07.098455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d519f-26e2-4c94-a411-847e1f27e25b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:07.117640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:07.133752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8f87c-688d-41ee-85cc-0887be457a61 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:07.142049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a091f-c4ec-4683-8c69-81a2626eea0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:07.142049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a091f-c4ec-4683-8c69-81a2626eea0b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.142317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a091f-c4ec-4683-8c69-81a2626eea0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.142522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a091f-c4ec-4683-8c69-81a2626eea0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.144324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a091f-c4ec-4683-8c69-81a2626eea0b None admin] RBAC: Adding query filter params (role.id=27d4ad049d8f4ff4a176581dea972512, user.id=7b03f4d117e84d1db43b54989c02efd9, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:07.144324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a091f-c4ec-4683-8c69-81a2626eea0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:07.147578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a091f-c4ec-4683-8c69-81a2626eea0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:07.151128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 143/281] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Jun 16 17:09:06 2022] GET /identity/v3/role_assignments?user.id=7b03f4d117e84d1db43b54989c02efd9&scope.system=all&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:07.152750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b2ff1-e7fe-4071-a567-11147fb24a73 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.152750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b2ff1-e7fe-4071-a567-11147fb24a73 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.152978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b2ff1-e7fe-4071-a567-11147fb24a73 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.156265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 144/282] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:07.162485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f037a-686b-4132-be00-78f1c5d80720 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.162907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f037a-686b-4132-be00-78f1c5d80720 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.163183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f037a-686b-4132-be00-78f1c5d80720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.182648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0d3b-f4c8-403b-9458-c1a4518249b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:07.183655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d3b-f4c8-403b-9458-c1a4518249b9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.183936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d3b-f4c8-403b-9458-c1a4518249b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.184257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e0d3b-f4c8-403b-9458-c1a4518249b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.186118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-650f037a-686b-4132-be00-78f1c5d80720 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:07.187576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-650f037a-686b-4132-be00-78f1c5d80720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:07.200073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442e0d3b-f4c8-403b-9458-c1a4518249b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-442e0d3b-f4c8-403b-9458-c1a4518249b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:07.211257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:07.212237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:07.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 139/283] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:07.219166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0d3b-f4c8-403b-9458-c1a4518249b9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:07.281665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 145/284] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:07.305433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f037a-686b-4132-be00-78f1c5d80720 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:07.342034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442e0d3b-f4c8-403b-9458-c1a4518249b9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:07.390840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:07.392274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.392709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.393135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.394063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:07.394513 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:07.396664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:07.409210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 140/285] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:07.414745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:07.430044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:07.442477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:07.517853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f037a-686b-4132-be00-78f1c5d80720 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:07.561882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f26ac-c64c-44d0-8eed-fb10f6c2616f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:07.562714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f26ac-c64c-44d0-8eed-fb10f6c2616f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.562991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f26ac-c64c-44d0-8eed-fb10f6c2616f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.563262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f26ac-c64c-44d0-8eed-fb10f6c2616f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.586052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f26ac-c64c-44d0-8eed-fb10f6c2616f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6a2f26ac-c64c-44d0-8eed-fb10f6c2616f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:07.596078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:07.597137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:07.597990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 146/286] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:07.606172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f26ac-c64c-44d0-8eed-fb10f6c2616f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:07.623134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fc21b4-b4be-4ec7-ba66-a2eb394b0d9b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:07.687964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2604c1-27bf-4483-9706-b54426ffe9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:07.688863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2604c1-27bf-4483-9706-b54426ffe9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.689144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2604c1-27bf-4483-9706-b54426ffe9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.689447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2604c1-27bf-4483-9706-b54426ffe9d2 None admin] PATH_INFO: `/v3/users/5ea00c3789304f3f826adeff3f959dad` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.690872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2604c1-27bf-4483-9706-b54426ffe9d2 None admin] RBAC: Authorizing `identity:get_user(user_id=5ea00c3789304f3f826adeff3f959dad)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:07.695888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2604c1-27bf-4483-9706-b54426ffe9d2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:07.698821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 141/287] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:07 2022] GET /identity/v3/users/5ea00c3789304f3f826adeff3f959dad => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:07.708096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2604c1-27bf-4483-9706-b54426ffe9d2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:07.781383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f26ac-c64c-44d0-8eed-fb10f6c2616f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:07.809045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:07.831864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:07.832975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.833487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.833966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.835292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:07.835802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:07.840338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:07.844311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:07.863545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 147/288] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:07.869544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:07.874439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2604c1-27bf-4483-9706-b54426ffe9d2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:07.909796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd21f9d-4b2b-4bc7-95af-37a8a659c7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:07.911063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd21f9d-4b2b-4bc7-95af-37a8a659c7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:07.911541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd21f9d-4b2b-4bc7-95af-37a8a659c7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:07.912013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd21f9d-4b2b-4bc7-95af-37a8a659c7d7 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:07.913573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd21f9d-4b2b-4bc7-95af-37a8a659c7d7 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:07.915296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd21f9d-4b2b-4bc7-95af-37a8a659c7d7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:07.917342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 142/289] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:07 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:07.923051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd21f9d-4b2b-4bc7-95af-37a8a659c7d7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:08.039509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58de0d-a516-4339-969b-52f8f85baaca None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:08.088480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4420850-28be-4c50-ba3e-386307769cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:08.089817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4420850-28be-4c50-ba3e-386307769cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:08.091469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4420850-28be-4c50-ba3e-386307769cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:08.092031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4420850-28be-4c50-ba3e-386307769cc9 None admin] PATH_INFO: `/v3/users/b40a4b06fb05405db1aeb2ef76daad03` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:08.094836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4420850-28be-4c50-ba3e-386307769cc9 None admin] RBAC: Authorizing `identity:get_user(user_id=b40a4b06fb05405db1aeb2ef76daad03)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:08.097825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4420850-28be-4c50-ba3e-386307769cc9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:08.103983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 148/290] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:07 2022] GET /identity/v3/users/b40a4b06fb05405db1aeb2ef76daad03 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:08.109270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4420850-28be-4c50-ba3e-386307769cc9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:08.133131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd21f9d-4b2b-4bc7-95af-37a8a659c7d7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:08.206013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67595f-658f-4fb1-8618-366d04a07a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:08.207325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67595f-658f-4fb1-8618-366d04a07a06 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:08.207824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67595f-658f-4fb1-8618-366d04a07a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:08.208763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67595f-658f-4fb1-8618-366d04a07a06 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496/users/5ea00c3789304f3f826adeff3f959dad/roles/70ff25a62a5149a98cdf09458ee3fa10` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:08.213448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67595f-658f-4fb1-8618-366d04a07a06 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc31ecc9189941fa909edb765d79c496, user_id=5ea00c3789304f3f826adeff3f959dad, role_id=70ff25a62a5149a98cdf09458ee3fa10)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:08.213564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:08.216266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67595f-658f-4fb1-8618-366d04a07a06 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:08.238081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 143/291] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:07 2022] PUT /identity/v3/projects/fc31ecc9189941fa909edb765d79c496/users/5ea00c3789304f3f826adeff3f959dad/roles/70ff25a62a5149a98cdf09458ee3fa10 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:09:08.283869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4420850-28be-4c50-ba3e-386307769cc9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:08.321716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c507915-e587-4f36-90f0-256e39ca93de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:08.322774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c507915-e587-4f36-90f0-256e39ca93de None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:08.323262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c507915-e587-4f36-90f0-256e39ca93de None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:08.323770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c507915-e587-4f36-90f0-256e39ca93de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:08.417529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c507915-e587-4f36-90f0-256e39ca93de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:08.418962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c507915-e587-4f36-90f0-256e39ca93de None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:08.494770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:08.543466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 149/292] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:08.549100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67595f-658f-4fb1-8618-366d04a07a06 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:08.560605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:08.670559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67595f-658f-4fb1-8618-366d04a07a06 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:08.721693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e8110-eeaf-4ef6-bfeb-c54450fc2304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:08.722648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e8110-eeaf-4ef6-bfeb-c54450fc2304 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:08.722963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e8110-eeaf-4ef6-bfeb-c54450fc2304 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:08.723239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e8110-eeaf-4ef6-bfeb-c54450fc2304 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:08.724482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e8110-eeaf-4ef6-bfeb-c54450fc2304 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:08.726070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e8110-eeaf-4ef6-bfeb-c54450fc2304 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:08.728319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 144/293] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:08 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:08.734268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c507915-e587-4f36-90f0-256e39ca93de None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:08.857051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c507915-e587-4f36-90f0-256e39ca93de None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:08.893770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc4d9d-474a-4633-974b-07cc446cfd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:08.894483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc4d9d-474a-4633-974b-07cc446cfd22 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:08.894768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc4d9d-474a-4633-974b-07cc446cfd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:08.895052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc4d9d-474a-4633-974b-07cc446cfd22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:08.895896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc4d9d-474a-4633-974b-07cc446cfd22 None admin] RBAC: Adding query filter params (role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=fc31ecc9189941fa909edb765d79c496, user.id=b40a4b06fb05405db1aeb2ef76daad03) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:08.896207 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc4d9d-474a-4633-974b-07cc446cfd22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:08.897815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc4d9d-474a-4633-974b-07cc446cfd22 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:08.904514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 150/294] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:08 2022] GET /identity/v3/role_assignments?user.id=b40a4b06fb05405db1aeb2ef76daad03&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:09.450978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cbcbe-6afd-4904-8197-3208c04cba31 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:09.451356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cbcbe-6afd-4904-8197-3208c04cba31 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:09.451533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00cbcbe-6afd-4904-8197-3208c04cba31 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:09.453033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 145/295] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:09.459080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f7e27-ed0e-4bbb-b97c-532816be827d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:09.459578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f7e27-ed0e-4bbb-b97c-532816be827d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:09.460101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f7e27-ed0e-4bbb-b97c-532816be827d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:09.485230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-797f7e27-ed0e-4bbb-b97c-532816be827d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:09.485540 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-797f7e27-ed0e-4bbb-b97c-532816be827d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:09.521568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:09.570939 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 151/296] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:09.594590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cbcbe-6afd-4904-8197-3208c04cba31 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:09.680965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:09.713642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00cbcbe-6afd-4904-8197-3208c04cba31 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:09.734033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc169ab-f887-4310-81d9-7d9eca0c0ccf None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:09.734353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc169ab-f887-4310-81d9-7d9eca0c0ccf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:09.734594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc169ab-f887-4310-81d9-7d9eca0c0ccf None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:09.736333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 152/297] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:09.742182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892a888-2217-4e3a-a11f-ec55eb6a64dd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:09.742456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892a888-2217-4e3a-a11f-ec55eb6a64dd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:09.742735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892a888-2217-4e3a-a11f-ec55eb6a64dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:09.752203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5f54d-aa8c-498f-bf1e-051e4541c514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:09.753056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5f54d-aa8c-498f-bf1e-051e4541c514 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:09.753372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5f54d-aa8c-498f-bf1e-051e4541c514 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:09.753651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5f54d-aa8c-498f-bf1e-051e4541c514 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:09.760763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a892a888-2217-4e3a-a11f-ec55eb6a64dd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:09.762188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a892a888-2217-4e3a-a11f-ec55eb6a64dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:09.764697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5f54d-aa8c-498f-bf1e-051e4541c514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-fad5f54d-aa8c-498f-bf1e-051e4541c514 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:09.772688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:09.774194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:09.775698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 146/298] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:09.780160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5f54d-aa8c-498f-bf1e-051e4541c514 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:09.837962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 153/299] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:09.839462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afbf7-6b63-4bb7-be57-238d58770c35 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:09.839744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afbf7-6b63-4bb7-be57-238d58770c35 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:09.840043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510afbf7-6b63-4bb7-be57-238d58770c35 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:09.845497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 154/300] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:09.846787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a79d8-855f-41ac-80d9-dbda52e2d2af None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:09.847067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a79d8-855f-41ac-80d9-dbda52e2d2af None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:09.847345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a79d8-855f-41ac-80d9-dbda52e2d2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:09.867502 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d99a79d8-855f-41ac-80d9-dbda52e2d2af None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:09.868716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d99a79d8-855f-41ac-80d9-dbda52e2d2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:09.901391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5f54d-aa8c-498f-bf1e-051e4541c514 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:09.937985 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 155/301] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:09.938828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:09.939768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:09.940271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:09.940742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:09.941702 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:09.942037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a79d8-855f-41ac-80d9-dbda52e2d2af None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:09.942398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:09.944126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:09.953484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:09.953976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 147/302] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:09.958715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:09.959279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:10.057649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a79d8-855f-41ac-80d9-dbda52e2d2af None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:10.067108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34db632-1fba-450d-950e-0fd298c48609 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:10.094299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d8c15-13d7-489f-a32f-96cc5b25dbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:10.095224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8c15-13d7-489f-a32f-96cc5b25dbec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.095499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8c15-13d7-489f-a32f-96cc5b25dbec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.095768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8c15-13d7-489f-a32f-96cc5b25dbec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.100835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27cbfb-3b9a-4a3a-8a89-dbb33e3a03ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:10.101754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27cbfb-3b9a-4a3a-8a89-dbb33e3a03ec None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.102076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27cbfb-3b9a-4a3a-8a89-dbb33e3a03ec None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.102296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27cbfb-3b9a-4a3a-8a89-dbb33e3a03ec None admin] PATH_INFO: `/v3/users/e7b80202128f4015a5d9a33cb17a4b5b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.107143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d8c15-13d7-489f-a32f-96cc5b25dbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-144d8c15-13d7-489f-a32f-96cc5b25dbec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:10.115236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:10.116132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:10.117145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:10.117808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27cbfb-3b9a-4a3a-8a89-dbb33e3a03ec None admin] RBAC: Authorizing `identity:get_user(user_id=e7b80202128f4015a5d9a33cb17a4b5b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:10.117808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27cbfb-3b9a-4a3a-8a89-dbb33e3a03ec None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:10.123904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 148/303] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:09 2022] GET /identity/v3/users/e7b80202128f4015a5d9a33cb17a4b5b => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:10.123904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27cbfb-3b9a-4a3a-8a89-dbb33e3a03ec None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:10.124118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 156/304] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:10.124118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d8c15-13d7-489f-a32f-96cc5b25dbec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:10.183399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:10.196587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:10.230601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27cbfb-3b9a-4a3a-8a89-dbb33e3a03ec None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:10.235589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d8c15-13d7-489f-a32f-96cc5b25dbec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:10.267812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae878e6-070b-4b62-8f0f-5a37b84cce16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:10.268966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae878e6-070b-4b62-8f0f-5a37b84cce16 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.269455 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae878e6-070b-4b62-8f0f-5a37b84cce16 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.269871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae878e6-070b-4b62-8f0f-5a37b84cce16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.271197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae878e6-070b-4b62-8f0f-5a37b84cce16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:10.271484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:10.272489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.272772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.273059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.273626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae878e6-070b-4b62-8f0f-5a37b84cce16 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:10.273889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:10.274202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:10.275580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:10.276217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 149/305] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:10.277126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae878e6-070b-4b62-8f0f-5a37b84cce16 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:10.289045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 157/306] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:10.294105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:10.397833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae878e6-070b-4b62-8f0f-5a37b84cce16 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:10.399886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481a36d-0b11-43b7-956b-183e4a9c6581 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:10.423075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:10.431991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:10.436962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:10.437651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.437930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.438204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.438464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a106aa2-02c9-4edb-8069-27ef7a89cbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:10.439042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] RBAC: Adding query filter params (role.id=70ff25a62a5149a98cdf09458ee3fa10, user.id=e7b80202128f4015a5d9a33cb17a4b5b, scope.system=all) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:10.439137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a106aa2-02c9-4edb-8069-27ef7a89cbc2 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.439361 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:10.439463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a106aa2-02c9-4edb-8069-27ef7a89cbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.439682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a106aa2-02c9-4edb-8069-27ef7a89cbc2 None admin] PATH_INFO: `/v3/groups` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.440381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a106aa2-02c9-4edb-8069-27ef7a89cbc2 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:10.440801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:10.441856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a106aa2-02c9-4edb-8069-27ef7a89cbc2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:10.446414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 150/307] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Jun 16 17:09:10 2022] GET /identity/v3/role_assignments?user.id=e7b80202128f4015a5d9a33cb17a4b5b&scope.system=all&role.id=70ff25a62a5149a98cdf09458ee3fa10 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:10.447118 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:10.475232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 158/308] 10.222.0.66 () {60 vars in 1134 bytes} [Thu Jun 16 17:09:10 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:09:10.587778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e71fe-582f-4308-87e6-71f1caeb1f73 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:10.629858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880ab09-0a57-4bcf-a37e-cc8d099a7df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:10.630964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880ab09-0a57-4bcf-a37e-cc8d099a7df4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.631247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880ab09-0a57-4bcf-a37e-cc8d099a7df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.631660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880ab09-0a57-4bcf-a37e-cc8d099a7df4 None admin] PATH_INFO: `/v3/users/7b03f4d117e84d1db43b54989c02efd9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.633528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880ab09-0a57-4bcf-a37e-cc8d099a7df4 None admin] RBAC: Authorizing `identity:get_user(user_id=7b03f4d117e84d1db43b54989c02efd9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:10.635072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1880ab09-0a57-4bcf-a37e-cc8d099a7df4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:10.636978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 151/309] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:10 2022] GET /identity/v3/users/7b03f4d117e84d1db43b54989c02efd9 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:10.643274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a106aa2-02c9-4edb-8069-27ef7a89cbc2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:10.713654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4665f-8cc8-4870-b2f7-53f8ee0ee30d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.713936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4665f-8cc8-4870-b2f7-53f8ee0ee30d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.714213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4665f-8cc8-4870-b2f7-53f8ee0ee30d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.715888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 152/310] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:10.720972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf910-987a-481f-83bd-418f63ecca11 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.721277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf910-987a-481f-83bd-418f63ecca11 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.721562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cf910-987a-481f-83bd-418f63ecca11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.742061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-810cf910-987a-481f-83bd-418f63ecca11 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:10.743684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-810cf910-987a-481f-83bd-418f63ecca11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:10.766786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a106aa2-02c9-4edb-8069-27ef7a89cbc2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:10.804499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:10.813389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5352726-e63d-4121-904e-b9f6d0377cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:10.814947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352726-e63d-4121-904e-b9f6d0377cf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:10.815828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352726-e63d-4121-904e-b9f6d0377cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:10.816532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5352726-e63d-4121-904e-b9f6d0377cf0 None admin] PATH_INFO: `/v3/system/users/7b03f4d117e84d1db43b54989c02efd9/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:10.819203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5352726-e63d-4121-904e-b9f6d0377cf0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7b03f4d117e84d1db43b54989c02efd9, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:10.821335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5352726-e63d-4121-904e-b9f6d0377cf0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:10.826915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 153/311] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:10.839996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 159/312] 10.222.0.66 () {58 vars in 1334 bytes} [Thu Jun 16 17:09:10 2022] PUT /identity/v3/system/users/7b03f4d117e84d1db43b54989c02efd9/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:10.851052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cf910-987a-481f-83bd-418f63ecca11 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:10.967249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cf910-987a-481f-83bd-418f63ecca11 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:11.006043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486892b2-0bcc-4859-8888-eb353936cf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:11.007292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486892b2-0bcc-4859-8888-eb353936cf77 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:11.007573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486892b2-0bcc-4859-8888-eb353936cf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:11.007847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486892b2-0bcc-4859-8888-eb353936cf77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:11.024915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486892b2-0bcc-4859-8888-eb353936cf77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-486892b2-0bcc-4859-8888-eb353936cf77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:11.033270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:11.034264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:11.035087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:11.043148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 154/313] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:11.045019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5352726-e63d-4121-904e-b9f6d0377cf0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:11.108763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:11.154734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5352726-e63d-4121-904e-b9f6d0377cf0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:11.188373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:11.189454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:11.189945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:11.190415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:11.191421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:11.191961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:11.193568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:11.203489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 160/314] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:11.208649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486892b2-0bcc-4859-8888-eb353936cf77 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:11.257271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:11.314259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486892b2-0bcc-4859-8888-eb353936cf77 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:11.349696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dbb81-6319-496f-bac0-45a2ce22c690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:11.351210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dbb81-6319-496f-bac0-45a2ce22c690 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:11.351688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dbb81-6319-496f-bac0-45a2ce22c690 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:11.352158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dbb81-6319-496f-bac0-45a2ce22c690 None admin] PATH_INFO: `/v3/users/5ea00c3789304f3f826adeff3f959dad` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:11.353620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dbb81-6319-496f-bac0-45a2ce22c690 None admin] RBAC: Authorizing `identity:get_user(user_id=5ea00c3789304f3f826adeff3f959dad)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:11.355319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dbb81-6319-496f-bac0-45a2ce22c690 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:11.357170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 155/315] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:11 2022] GET /identity/v3/users/5ea00c3789304f3f826adeff3f959dad => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:11.365410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:11.467725 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b8966-7d6c-46fa-8db6-54461f641334 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:11.484342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:11.502881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986383d-6fba-460b-8906-8d039c0a2a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:11.503564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986383d-6fba-460b-8906-8d039c0a2a35 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:11.503844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986383d-6fba-460b-8906-8d039c0a2a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:11.504137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986383d-6fba-460b-8906-8d039c0a2a35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:11.572393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986383d-6fba-460b-8906-8d039c0a2a35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:11.573829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986383d-6fba-460b-8906-8d039c0a2a35 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:11.684988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 161/316] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:11.685990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dbb81-6319-496f-bac0-45a2ce22c690 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:11.791586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dbb81-6319-496f-bac0-45a2ce22c690 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:11.804671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:11.825732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab62ec2-f3db-442a-b5f4-c76eba6006d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:11.826504 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62ec2-f3db-442a-b5f4-c76eba6006d3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:11.826782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62ec2-f3db-442a-b5f4-c76eba6006d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:11.827047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab62ec2-f3db-442a-b5f4-c76eba6006d3 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:11.828241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62ec2-f3db-442a-b5f4-c76eba6006d3 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:11.829730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab62ec2-f3db-442a-b5f4-c76eba6006d3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:11.831784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 156/317] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:11 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:11.837813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986383d-6fba-460b-8906-8d039c0a2a35 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:11.865775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:11.938230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986383d-6fba-460b-8906-8d039c0a2a35 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:11.972991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93a557b-39b2-4e21-b9ae-eeb569275593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:11.973703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a557b-39b2-4e21-b9ae-eeb569275593 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:11.973986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a557b-39b2-4e21-b9ae-eeb569275593 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:11.974261 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93a557b-39b2-4e21-b9ae-eeb569275593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:11.975139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a557b-39b2-4e21-b9ae-eeb569275593 None admin] RBAC: Adding query filter params (role.id=70ff25a62a5149a98cdf09458ee3fa10, scope.project.id=fc31ecc9189941fa909edb765d79c496, user.id=5ea00c3789304f3f826adeff3f959dad) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:11.975460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a557b-39b2-4e21-b9ae-eeb569275593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:11.976938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93a557b-39b2-4e21-b9ae-eeb569275593 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:11.983349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 162/318] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:11 2022] GET /identity/v3/role_assignments?user.id=5ea00c3789304f3f826adeff3f959dad&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=70ff25a62a5149a98cdf09458ee3fa10 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:12.724148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2f94f-4a8f-4cce-a9b3-6f42bd92309a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:12.724696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2f94f-4a8f-4cce-a9b3-6f42bd92309a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:12.724696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2f94f-4a8f-4cce-a9b3-6f42bd92309a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:12.726324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 157/319] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:12.731599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985eda8a-2194-4d14-b04a-71ca0d3a922d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:12.732065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985eda8a-2194-4d14-b04a-71ca0d3a922d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:12.732137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985eda8a-2194-4d14-b04a-71ca0d3a922d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:12.735573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03c6fa-ee11-4786-b588-a74cb9a403d5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:12.735847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03c6fa-ee11-4786-b588-a74cb9a403d5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:12.736126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03c6fa-ee11-4786-b588-a74cb9a403d5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:12.737545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 158/320] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:12.742774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71026e63-9c2d-4111-adf7-4f4d1040e95f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:12.743004 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71026e63-9c2d-4111-adf7-4f4d1040e95f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:12.743201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71026e63-9c2d-4111-adf7-4f4d1040e95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:12.750905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-985eda8a-2194-4d14-b04a-71ca0d3a922d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:12.752128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-985eda8a-2194-4d14-b04a-71ca0d3a922d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:12.760300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-71026e63-9c2d-4111-adf7-4f4d1040e95f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:12.761521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-71026e63-9c2d-4111-adf7-4f4d1040e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:12.826788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 163/321] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:12.837224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 159/322] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:12.849341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985eda8a-2194-4d14-b04a-71ca0d3a922d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:12.857052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71026e63-9c2d-4111-adf7-4f4d1040e95f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:12.862571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:12.956190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985eda8a-2194-4d14-b04a-71ca0d3a922d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:12.962799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71026e63-9c2d-4111-adf7-4f4d1040e95f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:12.973866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:12.993089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6bf57-74d5-4b12-8b64-011cacaa54fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:12.994210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6bf57-74d5-4b12-8b64-011cacaa54fb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:12.994694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6bf57-74d5-4b12-8b64-011cacaa54fb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:12.994968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6bf57-74d5-4b12-8b64-011cacaa54fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.001542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e0ecd-bbc7-45d7-a707-816d26ef93e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:13.002834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0ecd-bbc7-45d7-a707-816d26ef93e3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.003127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0ecd-bbc7-45d7-a707-816d26ef93e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.003404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e0ecd-bbc7-45d7-a707-816d26ef93e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.004585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e0ecd-bbc7-45d7-a707-816d26ef93e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:13.005407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6bf57-74d5-4b12-8b64-011cacaa54fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:13.006142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e0ecd-bbc7-45d7-a707-816d26ef93e3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:13.008226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 160/323] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:13.013431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e0ecd-bbc7-45d7-a707-816d26ef93e3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b6b6bf57-74d5-4b12-8b64-011cacaa54fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:13.013543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:13.014370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:13.015655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 164/324] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:13.021126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6bf57-74d5-4b12-8b64-011cacaa54fb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:13.096825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:13.125048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e0ecd-bbc7-45d7-a707-816d26ef93e3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:13.128724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6bf57-74d5-4b12-8b64-011cacaa54fb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:13.159653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14998cd9-7021-4c32-a5d5-f01236212042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:13.160303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14998cd9-7021-4c32-a5d5-f01236212042 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.160587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14998cd9-7021-4c32-a5d5-f01236212042 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.160867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14998cd9-7021-4c32-a5d5-f01236212042 None admin] PATH_INFO: `/v3/groups` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.161596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14998cd9-7021-4c32-a5d5-f01236212042 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:13.161723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a7c0c-8f3a-4fe8-a7b6-876e77887a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:13.162399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a7c0c-8f3a-4fe8-a7b6-876e77887a24 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.162927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a7c0c-8f3a-4fe8-a7b6-876e77887a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.163101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14998cd9-7021-4c32-a5d5-f01236212042 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:13.163440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a7c0c-8f3a-4fe8-a7b6-876e77887a24 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.164406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a7c0c-8f3a-4fe8-a7b6-876e77887a24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:13.164705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a7c0c-8f3a-4fe8-a7b6-876e77887a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:13.166442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a7c0c-8f3a-4fe8-a7b6-876e77887a24 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:13.177539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 165/325] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:13.178859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 161/326] 10.222.0.66 () {60 vars in 1134 bytes} [Thu Jun 16 17:09:13 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:09:13.178984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7c9a3-bb67-4571-9368-fcd0f4d7542b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.179091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7c9a3-bb67-4571-9368-fcd0f4d7542b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.179356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef7c9a3-bb67-4571-9368-fcd0f4d7542b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.180762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 166/327] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:13.183608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14998cd9-7021-4c32-a5d5-f01236212042 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:13.186120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cd7b-9cb0-42d7-8156-fcca9fb30cdb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.186369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cd7b-9cb0-42d7-8156-fcca9fb30cdb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.186682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9cd7b-9cb0-42d7-8156-fcca9fb30cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.207116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-56e9cd7b-9cb0-42d7-8156-fcca9fb30cdb None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:13.208340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-56e9cd7b-9cb0-42d7-8156-fcca9fb30cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:13.230477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:13.283965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 167/328] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:13.302749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14998cd9-7021-4c32-a5d5-f01236212042 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:13.308158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9cd7b-9cb0-42d7-8156-fcca9fb30cdb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:13.342455 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632d62a-b876-4429-a7c4-bdd07dad6e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:13.343395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632d62a-b876-4429-a7c4-bdd07dad6e66 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.343679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632d62a-b876-4429-a7c4-bdd07dad6e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.343956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632d62a-b876-4429-a7c4-bdd07dad6e66 None admin] PATH_INFO: `/v3/users/e7b80202128f4015a5d9a33cb17a4b5b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.345502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632d62a-b876-4429-a7c4-bdd07dad6e66 None admin] RBAC: Authorizing `identity:get_user(user_id=e7b80202128f4015a5d9a33cb17a4b5b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:13.346625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632d62a-b876-4429-a7c4-bdd07dad6e66 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:13.348532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 162/329] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:13 2022] GET /identity/v3/users/e7b80202128f4015a5d9a33cb17a4b5b => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:13.354773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632d62a-b876-4429-a7c4-bdd07dad6e66 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:13.375638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:13.421820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9cd7b-9cb0-42d7-8156-fcca9fb30cdb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:13.467729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b6176-0224-4e0d-b262-b2ea137f2c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:13.468557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b6176-0224-4e0d-b262-b2ea137f2c68 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.468842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b6176-0224-4e0d-b262-b2ea137f2c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.471905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b6176-0224-4e0d-b262-b2ea137f2c68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.485146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632d62a-b876-4429-a7c4-bdd07dad6e66 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:13.491469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b6176-0224-4e0d-b262-b2ea137f2c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:13.497914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-388b6176-0224-4e0d-b262-b2ea137f2c68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:13.499407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:13.500315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:13.501373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:13.503088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 168/330] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:13.507286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b6176-0224-4e0d-b262-b2ea137f2c68 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:13.536052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2627487-280c-4cc0-87ba-4c82172081dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:13.536861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2627487-280c-4cc0-87ba-4c82172081dd None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.537138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2627487-280c-4cc0-87ba-4c82172081dd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.537724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2627487-280c-4cc0-87ba-4c82172081dd None admin] PATH_INFO: `/v3/system/users/e7b80202128f4015a5d9a33cb17a4b5b/roles/70ff25a62a5149a98cdf09458ee3fa10` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.539388 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2627487-280c-4cc0-87ba-4c82172081dd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e7b80202128f4015a5d9a33cb17a4b5b, role_id=70ff25a62a5149a98cdf09458ee3fa10)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:13.541790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2627487-280c-4cc0-87ba-4c82172081dd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:13.553262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 163/331] 10.222.0.66 () {58 vars in 1334 bytes} [Thu Jun 16 17:09:13 2022] PUT /identity/v3/system/users/e7b80202128f4015a5d9a33cb17a4b5b/roles/70ff25a62a5149a98cdf09458ee3fa10 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:09:13.642379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b6176-0224-4e0d-b262-b2ea137f2c68 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:13.669239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:13.683370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:13.684087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.684362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.684651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.685481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:13.685787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:13.687227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:13.699212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 169/332] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:13.704250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2627487-280c-4cc0-87ba-4c82172081dd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:13.819006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2627487-280c-4cc0-87ba-4c82172081dd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:13.858213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304c0bd-3d0b-4c53-8b67-eb71b4cdee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:13.859110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304c0bd-3d0b-4c53-8b67-eb71b4cdee2b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:13.859395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304c0bd-3d0b-4c53-8b67-eb71b4cdee2b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:13.859672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304c0bd-3d0b-4c53-8b67-eb71b4cdee2b None admin] PATH_INFO: `/v3/users/7b03f4d117e84d1db43b54989c02efd9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:13.860938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304c0bd-3d0b-4c53-8b67-eb71b4cdee2b None admin] RBAC: Authorizing `identity:get_user(user_id=7b03f4d117e84d1db43b54989c02efd9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:13.862442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304c0bd-3d0b-4c53-8b67-eb71b4cdee2b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:13.864560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 164/333] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:13 2022] GET /identity/v3/users/7b03f4d117e84d1db43b54989c02efd9 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:13.871281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:13.992502 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0124b8-9ff3-400f-bc32-855e20be287b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:14.034834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b95543-3134-4f77-83c6-6518cb3014b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:14.035519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95543-3134-4f77-83c6-6518cb3014b9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:14.035816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95543-3134-4f77-83c6-6518cb3014b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:14.036092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95543-3134-4f77-83c6-6518cb3014b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:14.036963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b95543-3134-4f77-83c6-6518cb3014b9 None admin] RBAC: Adding query filter params (role.id=27d4ad049d8f4ff4a176581dea972512, user.id=7b03f4d117e84d1db43b54989c02efd9, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:14.037300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b95543-3134-4f77-83c6-6518cb3014b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:14.039024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b95543-3134-4f77-83c6-6518cb3014b9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:14.056112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 170/334] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Jun 16 17:09:13 2022] GET /identity/v3/role_assignments?user.id=7b03f4d117e84d1db43b54989c02efd9&scope.system=all&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:15.637403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651314-e11f-43d6-8677-e61e69d0870e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:15.638069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651314-e11f-43d6-8677-e61e69d0870e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:15.638069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00651314-e11f-43d6-8677-e61e69d0870e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:15.639405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 165/335] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:15.652335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fc6b9-2e47-4a95-b784-6c72369be48d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:15.652835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fc6b9-2e47-4a95-b784-6c72369be48d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:15.652937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fc6b9-2e47-4a95-b784-6c72369be48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:15.679308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0e5fc6b9-2e47-4a95-b784-6c72369be48d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:15.680536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0e5fc6b9-2e47-4a95-b784-6c72369be48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:15.730389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:15.770234 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 171/336] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:15.826808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651314-e11f-43d6-8677-e61e69d0870e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:15.879378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:15.954311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00651314-e11f-43d6-8677-e61e69d0870e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:15.999013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8baadf-bc6e-40e1-9117-c3b487f8ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:15.999897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8baadf-bc6e-40e1-9117-c3b487f8ef5a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.000185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8baadf-bc6e-40e1-9117-c3b487f8ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.000462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8baadf-bc6e-40e1-9117-c3b487f8ef5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.016292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8baadf-bc6e-40e1-9117-c3b487f8ef5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-cb8baadf-bc6e-40e1-9117-c3b487f8ef5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:16.024401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:16.025393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:16.026232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:16.030647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 166/337] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:16.032053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fc6b9-2e47-4a95-b784-6c72369be48d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:16.078975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602535-982d-42d5-a569-e3e0234e3b3a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.079631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602535-982d-42d5-a569-e3e0234e3b3a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.080109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8602535-982d-42d5-a569-e3e0234e3b3a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.084586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 167/338] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:16.091949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a72cf-5fc6-41f7-bbd5-9a1c9c30be92 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.092545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a72cf-5fc6-41f7-bbd5-9a1c9c30be92 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.093041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a72cf-5fc6-41f7-bbd5-9a1c9c30be92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.120475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-427a72cf-5fc6-41f7-bbd5-9a1c9c30be92 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:16.122134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-427a72cf-5fc6-41f7-bbd5-9a1c9c30be92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:16.220020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 168/339] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:16.225300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fc6b9-2e47-4a95-b784-6c72369be48d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:16.250342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a72cf-5fc6-41f7-bbd5-9a1c9c30be92 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:16.284836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:16.302810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:16.303643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.303935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.304216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.304986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:16.305330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:16.306750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:16.327529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 172/340] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:16.337862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:16.418376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:16.435229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a72cf-5fc6-41f7-bbd5-9a1c9c30be92 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:16.510119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a969ca9-e1ef-48f5-96cb-5e313df23ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:16.511405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a969ca9-e1ef-48f5-96cb-5e313df23ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.511895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a969ca9-e1ef-48f5-96cb-5e313df23ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.512374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a969ca9-e1ef-48f5-96cb-5e313df23ae3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.528980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a969ca9-e1ef-48f5-96cb-5e313df23ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1a969ca9-e1ef-48f5-96cb-5e313df23ae3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:16.540285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:16.541324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 17:09:16.542222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:16.542880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 169/341] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:16.545783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3919-1b30-4a12-af15-c37586dcc0fb None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.545783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3919-1b30-4a12-af15-c37586dcc0fb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.545783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d3919-1b30-4a12-af15-c37586dcc0fb None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.546057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 170/342] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:16.554078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d3919-1b30-4a12-af15-c37586dcc0fb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:16.599801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdd4be-9ed1-4fd2-bd8c-5a2f69cd456e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:16.646807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c37305-5497-417e-a564-b274d40c3e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:16.647538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c37305-5497-417e-a564-b274d40c3e07 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.647850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c37305-5497-417e-a564-b274d40c3e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.648100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c37305-5497-417e-a564-b274d40c3e07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.717556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:16.737968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d3919-1b30-4a12-af15-c37586dcc0fb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:16.751195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c37305-5497-417e-a564-b274d40c3e07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:16.752716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c37305-5497-417e-a564-b274d40c3e07 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:16.798147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:16.799094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b11c55-b9a4-4d0a-824d-177a39b30eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:16.799826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b11c55-b9a4-4d0a-824d-177a39b30eaf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.800108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b11c55-b9a4-4d0a-824d-177a39b30eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.800391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b11c55-b9a4-4d0a-824d-177a39b30eaf None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.801157 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b11c55-b9a4-4d0a-824d-177a39b30eaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:16.801497 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b11c55-b9a4-4d0a-824d-177a39b30eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:16.803122 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b11c55-b9a4-4d0a-824d-177a39b30eaf None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:16.822418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 171/343] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:16.823698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e603b-aab0-4299-80bf-91642c97959f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:16.824022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e603b-aab0-4299-80bf-91642c97959f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:16.824254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e603b-aab0-4299-80bf-91642c97959f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:16.858947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1d1e603b-aab0-4299-80bf-91642c97959f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:16.860678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1d1e603b-aab0-4299-80bf-91642c97959f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:16.869795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 173/344] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:09:16.874447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c37305-5497-417e-a564-b274d40c3e07 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:16.947210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 172/345] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:16.948213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e603b-aab0-4299-80bf-91642c97959f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:16.983342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c37305-5497-417e-a564-b274d40c3e07 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.022263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5571eb-8e69-4afe-9d7a-f56aa98754ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.023565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5571eb-8e69-4afe-9d7a-f56aa98754ff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.024064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5571eb-8e69-4afe-9d7a-f56aa98754ff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.024547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5571eb-8e69-4afe-9d7a-f56aa98754ff None admin] PATH_INFO: `/v3/users/e7b80202128f4015a5d9a33cb17a4b5b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.026069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5571eb-8e69-4afe-9d7a-f56aa98754ff None admin] RBAC: Authorizing `identity:get_user(user_id=e7b80202128f4015a5d9a33cb17a4b5b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:17.027789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5571eb-8e69-4afe-9d7a-f56aa98754ff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:17.029755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 174/346] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:09:16 2022] GET /identity/v3/users/e7b80202128f4015a5d9a33cb17a4b5b => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:17.031319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfff22-0365-4875-9c2c-6b038851041d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.031833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfff22-0365-4875-9c2c-6b038851041d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.032319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dfff22-0365-4875-9c2c-6b038851041d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.033798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 175/347] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:17.035953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfff22-0365-4875-9c2c-6b038851041d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:17.042460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:17.053193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:17.078643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e603b-aab0-4299-80bf-91642c97959f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.119674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c8900-502c-4faf-93e0-ff2c2cf8449f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.120561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c8900-502c-4faf-93e0-ff2c2cf8449f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.120938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c8900-502c-4faf-93e0-ff2c2cf8449f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.121232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c8900-502c-4faf-93e0-ff2c2cf8449f None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.122563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c8900-502c-4faf-93e0-ff2c2cf8449f None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:17.124331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c8900-502c-4faf-93e0-ff2c2cf8449f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:17.130815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 173/348] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:16 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:17.130815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c8900-502c-4faf-93e0-ff2c2cf8449f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:17.154405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dfff22-0365-4875-9c2c-6b038851041d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.189809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f758b-f4fe-4246-b7c2-28a8fdb31819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.191016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f758b-f4fe-4246-b7c2-28a8fdb31819 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.191517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f758b-f4fe-4246-b7c2-28a8fdb31819 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.191997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f758b-f4fe-4246-b7c2-28a8fdb31819 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.202991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f758b-f4fe-4246-b7c2-28a8fdb31819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-063f758b-f4fe-4246-b7c2-28a8fdb31819 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:17.212825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:17.213893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:17.214732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:17.217142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 176/349] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:09:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:17.218750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bebc4b-3825-4951-8d25-bfadcf58b683 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.219257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bebc4b-3825-4951-8d25-bfadcf58b683 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.219747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bebc4b-3825-4951-8d25-bfadcf58b683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.237442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c7bebc4b-3825-4951-8d25-bfadcf58b683 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:17.238949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7bebc4b-3825-4951-8d25-bfadcf58b683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:17.240542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c8900-502c-4faf-93e0-ff2c2cf8449f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.279124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.279845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.280128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.280405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.281278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] RBAC: Adding query filter params (role.id=70ff25a62a5149a98cdf09458ee3fa10, user.id=e7b80202128f4015a5d9a33cb17a4b5b, scope.system=all) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:17.281596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:17.283196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:17.299241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 174/350] 10.222.0.66 () {56 vars in 1308 bytes} [Thu Jun 16 17:09:17 2022] GET /identity/v3/role_assignments?user.id=e7b80202128f4015a5d9a33cb17a4b5b&scope.system=all&role.id=70ff25a62a5149a98cdf09458ee3fa10 => generated 510 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:17.299241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:17.302810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:17.313225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:17.315395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 177/351] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:17.316630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bebc4b-3825-4951-8d25-bfadcf58b683 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:17.425690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bebc4b-3825-4951-8d25-bfadcf58b683 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.447084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fb736-c0f7-4628-9aca-bd0ac0a756a0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.495489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739b4fc-b9a4-4ed1-8ceb-0a4f21e2122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.496674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b4fc-b9a4-4ed1-8ceb-0a4f21e2122f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.496966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b4fc-b9a4-4ed1-8ceb-0a4f21e2122f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.497275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739b4fc-b9a4-4ed1-8ceb-0a4f21e2122f None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.504233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739b4fc-b9a4-4ed1-8ceb-0a4f21e2122f None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:17.505769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739b4fc-b9a4-4ed1-8ceb-0a4f21e2122f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:17.508279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 175/352] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:17 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:17.509104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739b4fc-b9a4-4ed1-8ceb-0a4f21e2122f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:17.536195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.536881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.537475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.537959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.538922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:17.539389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:17.541008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:17.560641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 178/353] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:17.561883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:17.587710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:17.624658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739b4fc-b9a4-4ed1-8ceb-0a4f21e2122f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.663992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792acbf-7f59-408e-a0d8-35b3797b6360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.664854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792acbf-7f59-408e-a0d8-35b3797b6360 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.665144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792acbf-7f59-408e-a0d8-35b3797b6360 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.665448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792acbf-7f59-408e-a0d8-35b3797b6360 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.677262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792acbf-7f59-408e-a0d8-35b3797b6360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:17.680208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-d792acbf-7f59-408e-a0d8-35b3797b6360 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:17.686785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:17.687936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:17.688945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 176/354] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:17.693325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792acbf-7f59-408e-a0d8-35b3797b6360 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:17.762614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3566e9-0c36-4eed-8833-5e833fd26e63 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.810277 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.811032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.811319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.811601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:17.812454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] RBAC: Adding query filter params (group.id=d31c3498b9fa46db8fe60abd91e074d9, role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:17.812771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:17.814295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:17.821073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 179/355] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:17 2022] GET /identity/v3/role_assignments?group.id=d31c3498b9fa46db8fe60abd91e074d9&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:17.821984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:17.828777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792acbf-7f59-408e-a0d8-35b3797b6360 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:17.895105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:17.916927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1df34a-86a4-444b-86e8-0d42388404f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:17.917962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1df34a-86a4-444b-86e8-0d42388404f6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:17.918402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1df34a-86a4-444b-86e8-0d42388404f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:17.918832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1df34a-86a4-444b-86e8-0d42388404f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.024121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1df34a-86a4-444b-86e8-0d42388404f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:18.028199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1df34a-86a4-444b-86e8-0d42388404f6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:18.063401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe159733-e34c-4267-97d5-d6d477255325 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:18.091537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:18.118300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:18.119011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.119292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.119564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.120327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:18.120631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:18.122063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:18.133214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 180/356] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:18.142214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:18.163606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 177/357] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:18.165499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ecb5-99a8-4a14-b8f1-48a723e75a3a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.165950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ecb5-99a8-4a14-b8f1-48a723e75a3a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.166382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ecb5-99a8-4a14-b8f1-48a723e75a3a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.168089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 178/358] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:18.169737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3ecb5-99a8-4a14-b8f1-48a723e75a3a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:18.227880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:18.292226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e7729e-0ac3-4b75-9078-5a2aa4812782 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:18.319663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3ecb5-99a8-4a14-b8f1-48a723e75a3a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:18.338155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671599e-8138-4cc3-9ac5-656d128bf342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:18.339135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671599e-8138-4cc3-9ac5-656d128bf342 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.339470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671599e-8138-4cc3-9ac5-656d128bf342 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.339745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671599e-8138-4cc3-9ac5-656d128bf342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.363465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5650f0e-a15f-42fd-b495-3c1831fb5829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:18.364390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650f0e-a15f-42fd-b495-3c1831fb5829 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.364949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650f0e-a15f-42fd-b495-3c1831fb5829 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.365515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650f0e-a15f-42fd-b495-3c1831fb5829 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.367970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5650f0e-a15f-42fd-b495-3c1831fb5829 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:18.370695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5650f0e-a15f-42fd-b495-3c1831fb5829 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:18.372838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 179/359] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:18 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:18.378509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b23bb-a132-4ab7-b8d1-5891f18e8887 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.378945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b23bb-a132-4ab7-b8d1-5891f18e8887 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.379231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b23bb-a132-4ab7-b8d1-5891f18e8887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.401078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-737b23bb-a132-4ab7-b8d1-5891f18e8887 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:18.402762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-737b23bb-a132-4ab7-b8d1-5891f18e8887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:18.413704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:18.436040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671599e-8138-4cc3-9ac5-656d128bf342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:18.438298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671599e-8138-4cc3-9ac5-656d128bf342 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:18.487369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 180/360] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:18.488623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50c0f6-1c34-4eeb-bf40-58667a0be2d0 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.488906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50c0f6-1c34-4eeb-bf40-58667a0be2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.489217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e50c0f6-1c34-4eeb-bf40-58667a0be2d0 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.490602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 181/361] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:18.491591 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50c0f6-1c34-4eeb-bf40-58667a0be2d0 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:18.522737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:18.585372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 181/362] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:18.586805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b190a-a222-4b42-9e2a-291a5f7cbe61 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.587100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b190a-a222-4b42-9e2a-291a5f7cbe61 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.587374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b190a-a222-4b42-9e2a-291a5f7cbe61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.606678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0a3b190a-a222-4b42-9e2a-291a5f7cbe61 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:18.607854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a3b190a-a222-4b42-9e2a-291a5f7cbe61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:18.618879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e50c0f6-1c34-4eeb-bf40-58667a0be2d0 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:18.673124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd273af-29b5-48b2-aa46-ab9c5c3ef44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:18.674014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd273af-29b5-48b2-aa46-ab9c5c3ef44a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.674267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd273af-29b5-48b2-aa46-ab9c5c3ef44a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.674565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd273af-29b5-48b2-aa46-ab9c5c3ef44a None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.676115 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd273af-29b5-48b2-aa46-ab9c5c3ef44a None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:18.677765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd273af-29b5-48b2-aa46-ab9c5c3ef44a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:18.679607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 182/363] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:18 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:18.680321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd273af-29b5-48b2-aa46-ab9c5c3ef44a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:18.706965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 182/364] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:18.707952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b190a-a222-4b42-9e2a-291a5f7cbe61 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:18.725428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:18.819363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:18.833061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd273af-29b5-48b2-aa46-ab9c5c3ef44a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:18.859441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b190a-a222-4b42-9e2a-291a5f7cbe61 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:18.909087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a970-bed6-4f16-881d-7480710e1bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:18.909516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d2861-a627-46fa-9f9d-5444b97de5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:18.910013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a970-bed6-4f16-881d-7480710e1bde None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.910291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a970-bed6-4f16-881d-7480710e1bde None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.910565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c16a970-bed6-4f16-881d-7480710e1bde None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.910667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d2861-a627-46fa-9f9d-5444b97de5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:18.910917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d2861-a627-46fa-9f9d-5444b97de5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:18.911190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d2861-a627-46fa-9f9d-5444b97de5c6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:18.911784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a970-bed6-4f16-881d-7480710e1bde None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:18.913604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c16a970-bed6-4f16-881d-7480710e1bde None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:18.917058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 183/365] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:18 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:18.917058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a970-bed6-4f16-881d-7480710e1bde None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:18.927205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8d2861-a627-46fa-9f9d-5444b97de5c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-df8d2861-a627-46fa-9f9d-5444b97de5c6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:18.936772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:18.937826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:18.938652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:18.939843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 183/366] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:09:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:18.940931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d2861-a627-46fa-9f9d-5444b97de5c6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:19.045402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:19.080674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c16a970-bed6-4f16-881d-7480710e1bde None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:19.096735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d2861-a627-46fa-9f9d-5444b97de5c6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:19.133468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:19.134524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:19.134524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:19.134700 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:19.135501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] RBAC: Adding query filter params (group.id=d31c3498b9fa46db8fe60abd91e074d9, role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:19.135832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:19.137989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:19.140102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08569005-75a6-4b71-be23-b1b9b34b5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:19.140968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08569005-75a6-4b71-be23-b1b9b34b5051 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:19.142627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08569005-75a6-4b71-be23-b1b9b34b5051 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:19.142627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08569005-75a6-4b71-be23-b1b9b34b5051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:19.145624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 184/367] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:18 2022] GET /identity/v3/role_assignments?group.id=d31c3498b9fa46db8fe60abd91e074d9&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:19.146541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:19.161749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08569005-75a6-4b71-be23-b1b9b34b5051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-08569005-75a6-4b71-be23-b1b9b34b5051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:19.171690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:19.172719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 184/368] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:19.174066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08569005-75a6-4b71-be23-b1b9b34b5051 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:19.204454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:19.297886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6e2e7-ca8b-4b46-93a0-d9bb07656886 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:19.318305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08569005-75a6-4b71-be23-b1b9b34b5051 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:19.341098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e9821-301f-4671-b1a2-962ab77ecd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:19.342393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e9821-301f-4671-b1a2-962ab77ecd3d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:19.342973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e9821-301f-4671-b1a2-962ab77ecd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:19.343455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e9821-301f-4671-b1a2-962ab77ecd3d None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:19.345691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e9821-301f-4671-b1a2-962ab77ecd3d None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:19.347618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e9821-301f-4671-b1a2-962ab77ecd3d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:19.349695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 185/369] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:19 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:19.350889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e9821-301f-4671-b1a2-962ab77ecd3d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:19.364570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:19.384289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:19.384989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:19.385730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:19.385730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:19.386489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:19.386800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:19.388235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:19.400893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 185/370] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:19.403635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:19.506229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e9821-301f-4671-b1a2-962ab77ecd3d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:19.519025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:19.550753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324038b-839a-4d67-b6f0-faab3d6d9521 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:19.551127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:19.552189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:19.552707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:19.553286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:19.554620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:19.555142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:19.557650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:19.570961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 186/371] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:19.571900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:19.593216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:19.593918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:19.594208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:19.594491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:19.595340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] RBAC: Adding query filter params (group.id=d31c3498b9fa46db8fe60abd91e074d9, role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=fc31ecc9189941fa909edb765d79c496) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:19.596044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:19.597953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:19.607136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 186/372] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:19 2022] GET /identity/v3/role_assignments?group.id=d31c3498b9fa46db8fe60abd91e074d9&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:19.614963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:19.745012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:19.816304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c593a8-4d0b-4b09-897c-ee6655e0584d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:19.878859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8035f78-9263-4fd1-9855-8c197a99001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:19.879489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd67d3d-14a0-4e65-8394-fc4f9016fd4e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:19.879853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035f78-9263-4fd1-9855-8c197a99001c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:19.879853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035f78-9263-4fd1-9855-8c197a99001c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:19.880158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8035f78-9263-4fd1-9855-8c197a99001c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:19.935751 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dd4a4-8f4f-40ab-b2b7-6e3fd40c577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:19.936430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dd4a4-8f4f-40ab-b2b7-6e3fd40c577e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:19.937402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dd4a4-8f4f-40ab-b2b7-6e3fd40c577e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:19.937689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dd4a4-8f4f-40ab-b2b7-6e3fd40c577e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:19.955480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:19.977676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8035f78-9263-4fd1-9855-8c197a99001c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:19.977676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8035f78-9263-4fd1-9855-8c197a99001c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:20.059153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:20.061471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dd4a4-8f4f-40ab-b2b7-6e3fd40c577e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:20.066664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1dd4a4-8f4f-40ab-b2b7-6e3fd40c577e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:20.098839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 187/373] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:20.110676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8035f78-9263-4fd1-9855-8c197a99001c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:20.193991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 187/374] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:20.199874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dd4a4-8f4f-40ab-b2b7-6e3fd40c577e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:20.245568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:20.285368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8035f78-9263-4fd1-9855-8c197a99001c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:20.308231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dd4a4-8f4f-40ab-b2b7-6e3fd40c577e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:20.345050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b40a2c-932b-48db-85b6-82b35f21a08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:20.345878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b40a2c-932b-48db-85b6-82b35f21a08c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:20.346168 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b40a2c-932b-48db-85b6-82b35f21a08c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:20.346451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b40a2c-932b-48db-85b6-82b35f21a08c None admin] PATH_INFO: `/v3/projects/546ad1026cd44cbea571239c12f41a3f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:20.347671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b40a2c-932b-48db-85b6-82b35f21a08c None admin] RBAC: Authorizing `identity:get_project(project_id=546ad1026cd44cbea571239c12f41a3f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:20.349143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b40a2c-932b-48db-85b6-82b35f21a08c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:20.351264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 188/375] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:20 2022] GET /identity/v3/projects/546ad1026cd44cbea571239c12f41a3f => generated 344 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:20.352137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6cb92-d4a6-4149-a40c-acce806ddddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:20.353035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6cb92-d4a6-4149-a40c-acce806ddddf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:20.353349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6cb92-d4a6-4149-a40c-acce806ddddf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:20.353627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6cb92-d4a6-4149-a40c-acce806ddddf None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:20.354876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6cb92-d4a6-4149-a40c-acce806ddddf None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:20.356385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6cb92-d4a6-4149-a40c-acce806ddddf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:20.356564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b40a2c-932b-48db-85b6-82b35f21a08c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:20.358443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 188/376] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:20 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:20.374321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6cb92-d4a6-4149-a40c-acce806ddddf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:20.454568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:20.474562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b40a2c-932b-48db-85b6-82b35f21a08c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:20.490825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:20.515982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c37d44-81ab-4685-8287-784e1c7cdfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:20.516894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c37d44-81ab-4685-8287-784e1c7cdfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:20.517188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c37d44-81ab-4685-8287-784e1c7cdfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:20.517486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c37d44-81ab-4685-8287-784e1c7cdfe3 None admin] PATH_INFO: `/v3/groups/65554ceb43da47faa302f2e55fada658` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:20.523889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c37d44-81ab-4685-8287-784e1c7cdfe3 None admin] RBAC: Authorizing `identity:get_group(group_id=65554ceb43da47faa302f2e55fada658)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:20.525485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c37d44-81ab-4685-8287-784e1c7cdfe3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:20.528363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 189/377] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:20 2022] GET /identity/v3/groups/65554ceb43da47faa302f2e55fada658 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:20.530004 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b52d1e-d25b-460e-9790-9549124fd6a5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:20.530296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b52d1e-d25b-460e-9790-9549124fd6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:20.530572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b52d1e-d25b-460e-9790-9549124fd6a5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:20.532232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 190/378] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:20.537607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b52d1e-d25b-460e-9790-9549124fd6a5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:20.586826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6cb92-d4a6-4149-a40c-acce806ddddf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:20.639500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29446698-2222-4f6d-a9dc-88d10f1751a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:20.640737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29446698-2222-4f6d-a9dc-88d10f1751a1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:20.641446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29446698-2222-4f6d-a9dc-88d10f1751a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:20.641738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29446698-2222-4f6d-a9dc-88d10f1751a1 None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:20.643333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29446698-2222-4f6d-a9dc-88d10f1751a1 None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:20.645608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29446698-2222-4f6d-a9dc-88d10f1751a1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:20.647722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 189/379] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:20 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:20.648991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bcc9e-21df-4a6b-8817-a16d1cc8871c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:20.650155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bcc9e-21df-4a6b-8817-a16d1cc8871c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:20.650442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bcc9e-21df-4a6b-8817-a16d1cc8871c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:20.679949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-055bcc9e-21df-4a6b-8817-a16d1cc8871c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:20.680847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b52d1e-d25b-460e-9790-9549124fd6a5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:20.681210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-055bcc9e-21df-4a6b-8817-a16d1cc8871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:20.730162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:20.731426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:20.732834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:20.733308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:20.734298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] RBAC: Adding query filter params (group.id=65554ceb43da47faa302f2e55fada658, role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=546ad1026cd44cbea571239c12f41a3f) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:20.734803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:20.736884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:20.745592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 191/380] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:20 2022] GET /identity/v3/role_assignments?group.id=65554ceb43da47faa302f2e55fada658&scope.project.id=546ad1026cd44cbea571239c12f41a3f&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:20.746654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:20.792385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 190/381] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:20.796331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:20.832707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bcc9e-21df-4a6b-8817-a16d1cc8871c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:20.868717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669324c-1c58-4ee3-9911-f593e64ea482 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:20.910574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:20.912021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:20.912630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:20.913113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:20.914174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] RBAC: Adding query filter params (group.id=d31c3498b9fa46db8fe60abd91e074d9, role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=fc31ecc9189941fa909edb765d79c496) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:20.914396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:20.915653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:20.917592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:20.924978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 192/382] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:20 2022] GET /identity/v3/role_assignments?group.id=d31c3498b9fa46db8fe60abd91e074d9&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:21.021203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bcc9e-21df-4a6b-8817-a16d1cc8871c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:21.066902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe045-4b11-43ab-bf8e-97a851e8f30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:21.067759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe045-4b11-43ab-bf8e-97a851e8f30a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:21.068039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe045-4b11-43ab-bf8e-97a851e8f30a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:21.068319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe045-4b11-43ab-bf8e-97a851e8f30a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:21.088101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fe045-4b11-43ab-bf8e-97a851e8f30a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-904fe045-4b11-43ab-bf8e-97a851e8f30a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:21.095991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:21.096890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:21.097788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 191/383] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:21.105003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:21.234714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe1ed0-fc31-4b4c-9ad4-03c7194bd28d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:21.236746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:21.285332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:21.286284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:21.286724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:21.287149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:21.288164 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:21.288615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:21.290177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:21.305203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 193/384] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:21.305747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe045-4b11-43ab-bf8e-97a851e8f30a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:21.422597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:21.426588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe045-4b11-43ab-bf8e-97a851e8f30a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:21.471530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0eb5a6-9344-4f3f-a3ae-411e64815093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:21.472439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0eb5a6-9344-4f3f-a3ae-411e64815093 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:21.472721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0eb5a6-9344-4f3f-a3ae-411e64815093 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:21.472996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0eb5a6-9344-4f3f-a3ae-411e64815093 None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:21.474401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0eb5a6-9344-4f3f-a3ae-411e64815093 None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:21.476027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0eb5a6-9344-4f3f-a3ae-411e64815093 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:21.477799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 192/385] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:21 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:21.484001 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:21.605680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f804297-e475-4df9-9ffb-f09f341ef790 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:21.663314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c759fe-2513-4ded-8407-d8d46bcc5578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:21.664368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c759fe-2513-4ded-8407-d8d46bcc5578 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:21.664679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c759fe-2513-4ded-8407-d8d46bcc5578 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:21.664994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c759fe-2513-4ded-8407-d8d46bcc5578 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:21.666389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c759fe-2513-4ded-8407-d8d46bcc5578 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:21.667896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c759fe-2513-4ded-8407-d8d46bcc5578 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:21.673211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 194/386] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:21 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:21.682363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0eb5a6-9344-4f3f-a3ae-411e64815093 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:21.838204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3845d-586b-42f6-b8d5-e56849ec75c1 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:21.838398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3845d-586b-42f6-b8d5-e56849ec75c1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:21.838660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc3845d-586b-42f6-b8d5-e56849ec75c1 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:21.840388 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 195/387] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:21.846015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b07cbc-94b9-4897-a2bb-4ef660dba36f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:21.846246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b07cbc-94b9-4897-a2bb-4ef660dba36f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:21.846502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b07cbc-94b9-4897-a2bb-4ef660dba36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:21.857023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0eb5a6-9344-4f3f-a3ae-411e64815093 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:21.871123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f6b07cbc-94b9-4897-a2bb-4ef660dba36f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:21.872401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f6b07cbc-94b9-4897-a2bb-4ef660dba36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:21.903792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcc58b-0b46-412c-8753-676a0e955a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:21.904731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcc58b-0b46-412c-8753-676a0e955a34 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:21.905012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcc58b-0b46-412c-8753-676a0e955a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:21.905401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcc58b-0b46-412c-8753-676a0e955a34 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/groups/d31c3498b9fa46db8fe60abd91e074d9/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:21.908917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcc58b-0b46-412c-8753-676a0e955a34 None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd72df2adb4ac4a5cc614f9d1b9167, group_id=d31c3498b9fa46db8fe60abd91e074d9, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:21.910480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fcc58b-0b46-412c-8753-676a0e955a34 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:21.915258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:21.938243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 193/388] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Jun 16 17:09:21 2022] PUT /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/groups/d31c3498b9fa46db8fe60abd91e074d9/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:21.970355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 196/389] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:21.995190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcc58b-0b46-412c-8753-676a0e955a34 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:22.045593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:22.158944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcc58b-0b46-412c-8753-676a0e955a34 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:22.207772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6dae-e10a-47e1-9b14-11ffafe63717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:22.208718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6dae-e10a-47e1-9b14-11ffafe63717 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:22.209339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6dae-e10a-47e1-9b14-11ffafe63717 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:22.209339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b6dae-e10a-47e1-9b14-11ffafe63717 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:22.227240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b6dae-e10a-47e1-9b14-11ffafe63717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c30b6dae-e10a-47e1-9b14-11ffafe63717 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:22.237020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:22.237975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:22.238807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 194/390] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:09:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:22.246301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b07cbc-94b9-4897-a2bb-4ef660dba36f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:22.365969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b07cbc-94b9-4897-a2bb-4ef660dba36f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:22.414181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:22.414781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:22.415059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:22.415333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:22.416104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:22.416406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:22.417867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:22.434313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6dae-e10a-47e1-9b14-11ffafe63717 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:22.445195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 197/391] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:22.547817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:22.567941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b6dae-e10a-47e1-9b14-11ffafe63717 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:22.613145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9604612-fc85-49f4-8f89-d591b6dd7821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:22.614112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9604612-fc85-49f4-8f89-d591b6dd7821 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:22.614395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9604612-fc85-49f4-8f89-d591b6dd7821 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:22.614672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9604612-fc85-49f4-8f89-d591b6dd7821 None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:22.616644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9604612-fc85-49f4-8f89-d591b6dd7821 None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:22.618170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9604612-fc85-49f4-8f89-d591b6dd7821 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:22.620254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 195/392] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:22 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:22.625955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:22.668036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:22.779963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a72c08-b4f4-4308-9634-d545981aed45 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:22.822789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06638328-c7d2-4111-9656-63405d0722b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:22.824143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638328-c7d2-4111-9656-63405d0722b1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:22.824634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638328-c7d2-4111-9656-63405d0722b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:22.825127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06638328-c7d2-4111-9656-63405d0722b1 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:22.826764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06638328-c7d2-4111-9656-63405d0722b1 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:22.828451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06638328-c7d2-4111-9656-63405d0722b1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:22.831416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 198/393] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:22 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:22.838000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9604612-fc85-49f4-8f89-d591b6dd7821 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:22.971813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9604612-fc85-49f4-8f89-d591b6dd7821 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:23.016339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b0eeb-3e8b-4b39-80e3-bdb1fa087252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:23.017276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b0eeb-3e8b-4b39-80e3-bdb1fa087252 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:23.017565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b0eeb-3e8b-4b39-80e3-bdb1fa087252 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:23.017842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b0eeb-3e8b-4b39-80e3-bdb1fa087252 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/groups/d31c3498b9fa46db8fe60abd91e074d9/roles/774686ef0c4c42ba97f56e75b67d313e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:23.020204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b0eeb-3e8b-4b39-80e3-bdb1fa087252 None admin] RBAC: Authorizing `identity:create_grant(project_id=78bd72df2adb4ac4a5cc614f9d1b9167, group_id=d31c3498b9fa46db8fe60abd91e074d9, role_id=774686ef0c4c42ba97f56e75b67d313e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:23.021830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b0eeb-3e8b-4b39-80e3-bdb1fa087252 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:23.040241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 196/394] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Jun 16 17:09:22 2022] PUT /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167/groups/d31c3498b9fa46db8fe60abd91e074d9/roles/774686ef0c4c42ba97f56e75b67d313e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:23.255019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad926373-eff4-4bce-be29-e953a0bc08a9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:23.256244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad926373-eff4-4bce-be29-e953a0bc08a9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:23.256771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad926373-eff4-4bce-be29-e953a0bc08a9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:23.258482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 199/395] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:23.264399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ebe59-bb70-4351-ae58-c39c593775a7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:23.264695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ebe59-bb70-4351-ae58-c39c593775a7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:23.264978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072ebe59-bb70-4351-ae58-c39c593775a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:23.286760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-072ebe59-bb70-4351-ae58-c39c593775a7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:23.288335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-072ebe59-bb70-4351-ae58-c39c593775a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:23.300170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:23.369236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 197/396] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:23.411794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad926373-eff4-4bce-be29-e953a0bc08a9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:23.533472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:23.550796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad926373-eff4-4bce-be29-e953a0bc08a9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:23.594782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dceea-c80e-4605-97d9-24bc18450fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:23.596030 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dceea-c80e-4605-97d9-24bc18450fdf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:23.596527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dceea-c80e-4605-97d9-24bc18450fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:23.597040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dceea-c80e-4605-97d9-24bc18450fdf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:23.616219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01dceea-c80e-4605-97d9-24bc18450fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-d01dceea-c80e-4605-97d9-24bc18450fdf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:23.624377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:23.625321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:23.626143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:23.629106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 200/397] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:23.636630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ebe59-bb70-4351-ae58-c39c593775a7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:23.665238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fabbef-32ff-4a80-bdeb-4285462ce3a8 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:23.665894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fabbef-32ff-4a80-bdeb-4285462ce3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:23.666486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fabbef-32ff-4a80-bdeb-4285462ce3a8 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:23.668116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 201/398] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:23.673730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea1840-965d-42dc-8ca0-9a727be51368 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:23.674284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea1840-965d-42dc-8ca0-9a727be51368 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:23.674753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea1840-965d-42dc-8ca0-9a727be51368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:23.693136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-68ea1840-965d-42dc-8ca0-9a727be51368 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:23.695265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-68ea1840-965d-42dc-8ca0-9a727be51368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:23.790083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 202/399] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:23.833197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072ebe59-bb70-4351-ae58-c39c593775a7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:23.838928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea1840-965d-42dc-8ca0-9a727be51368 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:23.908633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:23.909779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:23.911427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:23.911922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:23.913976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:23.914492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:23.918833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:23.936389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 198/400] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:23.941603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:23.953634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:23.981199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:24.008371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea1840-965d-42dc-8ca0-9a727be51368 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:24.056868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c2ac7-6da3-4334-b1ff-dc33c0a8551c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:24.082955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144015-700c-4c03-afe6-815d17b233fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:24.083805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144015-700c-4c03-afe6-815d17b233fd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.084146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144015-700c-4c03-afe6-815d17b233fd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.084661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144015-700c-4c03-afe6-815d17b233fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.096165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b144015-700c-4c03-afe6-815d17b233fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3b144015-700c-4c03-afe6-815d17b233fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:24.104596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:24.105579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:24.106877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 203/401] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:09:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:24.114301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144015-700c-4c03-afe6-815d17b233fd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:24.116406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f5432-57a1-4350-bf3b-4ddf611f2f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:24.117372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f5432-57a1-4350-bf3b-4ddf611f2f59 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.117659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f5432-57a1-4350-bf3b-4ddf611f2f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.117935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f5432-57a1-4350-bf3b-4ddf611f2f59 None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.119575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f5432-57a1-4350-bf3b-4ddf611f2f59 None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:24.121075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f5432-57a1-4350-bf3b-4ddf611f2f59 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:24.125927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 199/402] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:23 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:24.137423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f5432-57a1-4350-bf3b-4ddf611f2f59 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:24.273578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:24.328072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f5432-57a1-4350-bf3b-4ddf611f2f59 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:24.339256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144015-700c-4c03-afe6-815d17b233fd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:24.367962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e2282-808f-41a3-92ff-554c3601a656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:24.368940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e2282-808f-41a3-92ff-554c3601a656 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.369209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e2282-808f-41a3-92ff-554c3601a656 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.369504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e2282-808f-41a3-92ff-554c3601a656 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.370794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e2282-808f-41a3-92ff-554c3601a656 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:24.372292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e2282-808f-41a3-92ff-554c3601a656 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:24.374760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 200/403] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:24 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:24.378172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef4955-f646-4c0e-adc0-3736baa3058c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.378464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef4955-f646-4c0e-adc0-3736baa3058c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.378749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef4955-f646-4c0e-adc0-3736baa3058c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.381028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 201/404] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:24.389673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef4955-f646-4c0e-adc0-3736baa3058c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:24.391338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a75ce3-3486-4ec9-a227-afc29ac95f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:24.392096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a75ce3-3486-4ec9-a227-afc29ac95f42 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.392383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a75ce3-3486-4ec9-a227-afc29ac95f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.392664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a75ce3-3486-4ec9-a227-afc29ac95f42 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.393495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a75ce3-3486-4ec9-a227-afc29ac95f42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:24.393916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a75ce3-3486-4ec9-a227-afc29ac95f42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:24.395426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a75ce3-3486-4ec9-a227-afc29ac95f42 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:24.405936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 204/405] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:24.411853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7ed78-a3e3-4263-867a-0fded06e47dc None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.412250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7ed78-a3e3-4263-867a-0fded06e47dc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.412696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f7ed78-a3e3-4263-867a-0fded06e47dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.419761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:24.437462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c2f7ed78-a3e3-4263-867a-0fded06e47dc None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:24.439526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c2f7ed78-a3e3-4263-867a-0fded06e47dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:24.502024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef4955-f646-4c0e-adc0-3736baa3058c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:24.522211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 205/406] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:24.523486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7ed78-a3e3-4263-867a-0fded06e47dc None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:24.551270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179b43f-d6ac-4d41-bac7-4083a2033363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:24.552400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b43f-d6ac-4d41-bac7-4083a2033363 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.552581 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b43f-d6ac-4d41-bac7-4083a2033363 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.552919 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179b43f-d6ac-4d41-bac7-4083a2033363 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496/groups/d31c3498b9fa46db8fe60abd91e074d9/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.562744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179b43f-d6ac-4d41-bac7-4083a2033363 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc31ecc9189941fa909edb765d79c496, group_id=d31c3498b9fa46db8fe60abd91e074d9, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:24.564337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179b43f-d6ac-4d41-bac7-4083a2033363 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:24.585427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 202/407] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Jun 16 17:09:24 2022] PUT /identity/v3/projects/fc31ecc9189941fa909edb765d79c496/groups/d31c3498b9fa46db8fe60abd91e074d9/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:24.586642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179b43f-d6ac-4d41-bac7-4083a2033363 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:24.594617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:24.693298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f7ed78-a3e3-4263-867a-0fded06e47dc None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:24.742556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018d5d0-963e-4c31-860b-2c76cd4da5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:24.743440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018d5d0-963e-4c31-860b-2c76cd4da5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.743725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018d5d0-963e-4c31-860b-2c76cd4da5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.744003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018d5d0-963e-4c31-860b-2c76cd4da5f3 None admin] PATH_INFO: `/v3/groups/65554ceb43da47faa302f2e55fada658` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.746238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018d5d0-963e-4c31-860b-2c76cd4da5f3 None admin] RBAC: Authorizing `identity:get_group(group_id=65554ceb43da47faa302f2e55fada658)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:24.748042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018d5d0-963e-4c31-860b-2c76cd4da5f3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:24.751182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 206/408] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:24 2022] GET /identity/v3/groups/65554ceb43da47faa302f2e55fada658 => generated 229 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:24.753320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31df526-51f4-47b1-b25b-5f31ddc49a8d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.753621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31df526-51f4-47b1-b25b-5f31ddc49a8d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.754143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31df526-51f4-47b1-b25b-5f31ddc49a8d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.756223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 207/409] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:24.757262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31df526-51f4-47b1-b25b-5f31ddc49a8d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:24.761606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:24.766943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179b43f-d6ac-4d41-bac7-4083a2033363 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:24.823615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8136f4d-bd2a-4c12-997f-f627c5c83a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:24.825016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8136f4d-bd2a-4c12-997f-f627c5c83a20 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.825470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8136f4d-bd2a-4c12-997f-f627c5c83a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.825914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8136f4d-bd2a-4c12-997f-f627c5c83a20 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.840665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8136f4d-bd2a-4c12-997f-f627c5c83a20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f8136f4d-bd2a-4c12-997f-f627c5c83a20 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:24.852019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:24.852969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:24.853832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:24.855239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 203/410] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:09:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:24.857992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe5639-d637-4a67-86c5-3e85b5f397c2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.858485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe5639-d637-4a67-86c5-3e85b5f397c2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.858984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabe5639-d637-4a67-86c5-3e85b5f397c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.882423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fabe5639-d637-4a67-86c5-3e85b5f397c2 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:24.884425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fabe5639-d637-4a67-86c5-3e85b5f397c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:24.885810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31df526-51f4-47b1-b25b-5f31ddc49a8d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:24.924442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492be14c-3fd7-4eba-a116-f7e7d05982a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:24.925370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492be14c-3fd7-4eba-a116-f7e7d05982a2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:24.925661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492be14c-3fd7-4eba-a116-f7e7d05982a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:24.925938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492be14c-3fd7-4eba-a116-f7e7d05982a2 None admin] PATH_INFO: `/v3/projects/546ad1026cd44cbea571239c12f41a3f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:24.927155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492be14c-3fd7-4eba-a116-f7e7d05982a2 None admin] RBAC: Authorizing `identity:get_project(project_id=546ad1026cd44cbea571239c12f41a3f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:24.928897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492be14c-3fd7-4eba-a116-f7e7d05982a2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:24.931084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 208/411] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:24 2022] GET /identity/v3/projects/546ad1026cd44cbea571239c12f41a3f => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:24.931727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492be14c-3fd7-4eba-a116-f7e7d05982a2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:24.961694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:24.980139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 204/412] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:24.981352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe5639-d637-4a67-86c5-3e85b5f397c2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:25.030927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:25.069273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492be14c-3fd7-4eba-a116-f7e7d05982a2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:25.104812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabe5639-d637-4a67-86c5-3e85b5f397c2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:25.106697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:25.107383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.107792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.107913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.108684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:25.109008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:25.110588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:25.119553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 209/413] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:25.120197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:25.147832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8481b-9d02-4d4d-8c7b-92a763825b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:25.149069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8481b-9d02-4d4d-8c7b-92a763825b34 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.149446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8481b-9d02-4d4d-8c7b-92a763825b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.150010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8481b-9d02-4d4d-8c7b-92a763825b34 None admin] PATH_INFO: `/v3/projects/546ad1026cd44cbea571239c12f41a3f/groups/65554ceb43da47faa302f2e55fada658/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.153083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8481b-9d02-4d4d-8c7b-92a763825b34 None admin] RBAC: Authorizing `identity:create_grant(project_id=546ad1026cd44cbea571239c12f41a3f, group_id=65554ceb43da47faa302f2e55fada658, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:25.155350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8481b-9d02-4d4d-8c7b-92a763825b34 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:25.172611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 205/414] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Jun 16 17:09:24 2022] PUT /identity/v3/projects/546ad1026cd44cbea571239c12f41a3f/groups/65554ceb43da47faa302f2e55fada658/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:25.173829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8481b-9d02-4d4d-8c7b-92a763825b34 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:25.240201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d70353-aabd-4f8a-91a0-c5e50d8f6f19 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:25.254302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:25.274954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:25.282694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb947243-2857-4233-834e-7a9444fd3864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:25.283756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb947243-2857-4233-834e-7a9444fd3864 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.284203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb947243-2857-4233-834e-7a9444fd3864 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.284721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb947243-2857-4233-834e-7a9444fd3864 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.297222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb947243-2857-4233-834e-7a9444fd3864 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-fb947243-2857-4233-834e-7a9444fd3864 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:25.305909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:25.307069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:25.308026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:25.310340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 210/415] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:25.316145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb947243-2857-4233-834e-7a9444fd3864 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:25.317423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8481b-9d02-4d4d-8c7b-92a763825b34 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:25.428237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2465c31-41db-425d-b118-1492e22c8a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:25.429085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2465c31-41db-425d-b118-1492e22c8a64 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.429423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2465c31-41db-425d-b118-1492e22c8a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.429702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2465c31-41db-425d-b118-1492e22c8a64 None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.431334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2465c31-41db-425d-b118-1492e22c8a64 None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:25.432957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2465c31-41db-425d-b118-1492e22c8a64 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:25.435649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 206/416] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:25 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:25.443366 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2465c31-41db-425d-b118-1492e22c8a64 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:25.474084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb947243-2857-4233-834e-7a9444fd3864 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:25.564303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2465c31-41db-425d-b118-1492e22c8a64 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:25.570844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:25.571852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.572430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.572946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.573954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:25.574468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:25.576094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:25.586535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:25.593870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 211/417] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:25.602162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:25.607165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee4f1e-bb3a-4b6c-b425-f905680683b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:25.608323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee4f1e-bb3a-4b6c-b425-f905680683b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.608855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee4f1e-bb3a-4b6c-b425-f905680683b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.609412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee4f1e-bb3a-4b6c-b425-f905680683b8 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.610833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee4f1e-bb3a-4b6c-b425-f905680683b8 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:25.612494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee4f1e-bb3a-4b6c-b425-f905680683b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:25.614854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 207/418] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:25 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:25.620843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee4f1e-bb3a-4b6c-b425-f905680683b8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:25.702214 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:25.734369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee4f1e-bb3a-4b6c-b425-f905680683b8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:25.785328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330274d-2ee3-4fc1-b7df-27b4ec236214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:25.786357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330274d-2ee3-4fc1-b7df-27b4ec236214 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.786950 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330274d-2ee3-4fc1-b7df-27b4ec236214 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.787349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330274d-2ee3-4fc1-b7df-27b4ec236214 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496/groups/d31c3498b9fa46db8fe60abd91e074d9/roles/774686ef0c4c42ba97f56e75b67d313e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.790089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c6ce2-6ad5-4e80-864d-763ed3814fab None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:25.790208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330274d-2ee3-4fc1-b7df-27b4ec236214 None admin] RBAC: Authorizing `identity:create_grant(project_id=fc31ecc9189941fa909edb765d79c496, group_id=d31c3498b9fa46db8fe60abd91e074d9, role_id=774686ef0c4c42ba97f56e75b67d313e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:25.791712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330274d-2ee3-4fc1-b7df-27b4ec236214 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:25.810278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 208/419] 10.222.0.66 () {58 vars in 1442 bytes} [Thu Jun 16 17:09:25 2022] PUT /identity/v3/projects/fc31ecc9189941fa909edb765d79c496/groups/d31c3498b9fa46db8fe60abd91e074d9/roles/774686ef0c4c42ba97f56e75b67d313e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:09:25.817333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e511-f343-4905-b924-9c8b8908a772 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.817333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e511-f343-4905-b924-9c8b8908a772 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.817333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6e511-f343-4905-b924-9c8b8908a772 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.818373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 209/420] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:25.820580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15224c57-3e1d-4c1c-a027-896b6050bdf6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.820913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15224c57-3e1d-4c1c-a027-896b6050bdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.821218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15224c57-3e1d-4c1c-a027-896b6050bdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.843247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-15224c57-3e1d-4c1c-a027-896b6050bdf6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:25.844744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-15224c57-3e1d-4c1c-a027-896b6050bdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:25.866091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701c2d5-ecd6-4823-be2d-99a108c2728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:25.867191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701c2d5-ecd6-4823-be2d-99a108c2728b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:25.867703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701c2d5-ecd6-4823-be2d-99a108c2728b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:25.868241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a701c2d5-ecd6-4823-be2d-99a108c2728b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:25.922193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:25.972557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 210/421] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:25.992133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701c2d5-ecd6-4823-be2d-99a108c2728b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:25.994037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a701c2d5-ecd6-4823-be2d-99a108c2728b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:26.008712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15224c57-3e1d-4c1c-a027-896b6050bdf6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:26.113662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:26.162578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15224c57-3e1d-4c1c-a027-896b6050bdf6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:26.166635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 212/422] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:26.176798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701c2d5-ecd6-4823-be2d-99a108c2728b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:26.234537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b5e32-6e04-4168-ae9f-e936a1273ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:26.235352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5e32-6e04-4168-ae9f-e936a1273ccb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:26.235636 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5e32-6e04-4168-ae9f-e936a1273ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:26.235911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b5e32-6e04-4168-ae9f-e936a1273ccb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:26.261942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b5e32-6e04-4168-ae9f-e936a1273ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e58b5e32-6e04-4168-ae9f-e936a1273ccb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:26.277486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:26.278466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:26.279276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:26.281837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 211/423] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:09:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:26.284545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b5e32-6e04-4168-ae9f-e936a1273ccb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:26.361016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:26.367435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a701c2d5-ecd6-4823-be2d-99a108c2728b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:26.426608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b5e32-6e04-4168-ae9f-e936a1273ccb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:26.460729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19161ca-35e2-4a6f-abd7-e313d9e7ad62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:26.461665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19161ca-35e2-4a6f-abd7-e313d9e7ad62 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:26.461951 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19161ca-35e2-4a6f-abd7-e313d9e7ad62 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:26.462230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19161ca-35e2-4a6f-abd7-e313d9e7ad62 None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:26.464755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19161ca-35e2-4a6f-abd7-e313d9e7ad62 None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:26.466496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19161ca-35e2-4a6f-abd7-e313d9e7ad62 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:26.468430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 213/424] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:26 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:26.478027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19161ca-35e2-4a6f-abd7-e313d9e7ad62 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:26.501682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:26.502389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:26.504459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:26.504745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:26.505566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:26.506213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:26.509134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:26.527477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 212/425] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:26.530747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:26.534284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:26.644764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19161ca-35e2-4a6f-abd7-e313d9e7ad62 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:26.683517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e736-db26-4ecc-b549-ced454dda54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:26.684656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e736-db26-4ecc-b549-ced454dda54c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:26.685000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e736-db26-4ecc-b549-ced454dda54c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:26.685314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3e736-db26-4ecc-b549-ced454dda54c None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:26.686557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3e736-db26-4ecc-b549-ced454dda54c None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:26.688622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3e736-db26-4ecc-b549-ced454dda54c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:26.690987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 214/426] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:26 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:26.697978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e736-db26-4ecc-b549-ced454dda54c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:26.768335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:26.772585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc18be4-f628-4637-9fd6-5f755d92a659 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:26.822859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fc15e-786a-49fb-9c7e-bf959a92565b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:26.823656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fc15e-786a-49fb-9c7e-bf959a92565b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:26.823961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fc15e-786a-49fb-9c7e-bf959a92565b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:26.824225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fc15e-786a-49fb-9c7e-bf959a92565b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:26.842974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:26.879409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3e736-db26-4ecc-b549-ced454dda54c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:26.901913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fc15e-786a-49fb-9c7e-bf959a92565b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:26.903756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fc15e-786a-49fb-9c7e-bf959a92565b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:26.956239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83790713-b64f-46ff-88a7-b1a4d8e64a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:26.956933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83790713-b64f-46ff-88a7-b1a4d8e64a48 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:26.957236 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83790713-b64f-46ff-88a7-b1a4d8e64a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:26.957517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83790713-b64f-46ff-88a7-b1a4d8e64a48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:26.958380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83790713-b64f-46ff-88a7-b1a4d8e64a48 None admin] RBAC: Adding query filter params (group.id=d31c3498b9fa46db8fe60abd91e074d9, role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:26.958700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83790713-b64f-46ff-88a7-b1a4d8e64a48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:26.960667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83790713-b64f-46ff-88a7-b1a4d8e64a48 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:26.967581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 215/427] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:26 2022] GET /identity/v3/role_assignments?group.id=d31c3498b9fa46db8fe60abd91e074d9&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 611 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:26.995371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49b7ff-daf2-40cb-92d4-4740a6fcd816 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:26.995701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49b7ff-daf2-40cb-92d4-4740a6fcd816 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:26.995950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49b7ff-daf2-40cb-92d4-4740a6fcd816 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:26.997620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 216/428] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:27.010402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e551a1-10f1-4393-9893-df60a345602a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.010716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e551a1-10f1-4393-9893-df60a345602a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.010974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e551a1-10f1-4393-9893-df60a345602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.047413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a0e551a1-10f1-4393-9893-df60a345602a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:27.049215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a0e551a1-10f1-4393-9893-df60a345602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:27.066978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:27.073702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 213/429] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:09:27.086347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fc15e-786a-49fb-9c7e-bf959a92565b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:27.186285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 217/430] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:27.211381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e551a1-10f1-4393-9893-df60a345602a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:27.249910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fc15e-786a-49fb-9c7e-bf959a92565b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:27.289017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcb195-7d44-499c-83c9-214e42426b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:27.290021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb195-7d44-499c-83c9-214e42426b1c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.290306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb195-7d44-499c-83c9-214e42426b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.290586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb195-7d44-499c-83c9-214e42426b1c None admin] PATH_INFO: `/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.291953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcb195-7d44-499c-83c9-214e42426b1c None admin] RBAC: Authorizing `identity:get_project(project_id=78bd72df2adb4ac4a5cc614f9d1b9167)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:27.293569 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcb195-7d44-499c-83c9-214e42426b1c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:27.295654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 214/431] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:27 2022] GET /identity/v3/projects/78bd72df2adb4ac4a5cc614f9d1b9167 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:27.302903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcb195-7d44-499c-83c9-214e42426b1c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:27.316210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:27.334260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e551a1-10f1-4393-9893-df60a345602a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:27.371895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31b4b5-0c58-47af-a7fa-fe37b62849b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:27.372902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31b4b5-0c58-47af-a7fa-fe37b62849b1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.373219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31b4b5-0c58-47af-a7fa-fe37b62849b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.373539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31b4b5-0c58-47af-a7fa-fe37b62849b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.380646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:27.387395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31b4b5-0c58-47af-a7fa-fe37b62849b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-bb31b4b5-0c58-47af-a7fa-fe37b62849b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:27.395427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:27.396382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 17:09:27.397871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:27.401915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 218/432] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:27.403151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31b4b5-0c58-47af-a7fa-fe37b62849b1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:27.419503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcb195-7d44-499c-83c9-214e42426b1c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:27.462407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8e23b-c7b3-45ae-8acf-7971ae28f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:27.463310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8e23b-c7b3-45ae-8acf-7971ae28f312 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.463596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8e23b-c7b3-45ae-8acf-7971ae28f312 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.463872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8e23b-c7b3-45ae-8acf-7971ae28f312 None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.467149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8e23b-c7b3-45ae-8acf-7971ae28f312 None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:27.467570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8e23b-c7b3-45ae-8acf-7971ae28f312 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:27.469542 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 215/433] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:27 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:27.478142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8e23b-c7b3-45ae-8acf-7971ae28f312 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:27.551234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31b4b5-0c58-47af-a7fa-fe37b62849b1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:27.613851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:27.614086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8e23b-c7b3-45ae-8acf-7971ae28f312 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:27.615369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.616294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.617804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.618266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] RBAC: Adding query filter params (name=member) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:27.618584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:27.620152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:27.632499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 219/434] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:09:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:27.640759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:27.650279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:27.670620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed557ce3-c746-4b81-ae86-21c2f134ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:27.671793 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed557ce3-c746-4b81-ae86-21c2f134ada4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.672504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed557ce3-c746-4b81-ae86-21c2f134ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.672932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed557ce3-c746-4b81-ae86-21c2f134ada4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.674536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed557ce3-c746-4b81-ae86-21c2f134ada4 None admin] RBAC: Adding query filter params (group.id=d31c3498b9fa46db8fe60abd91e074d9, role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:27.675001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed557ce3-c746-4b81-ae86-21c2f134ada4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:27.677407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed557ce3-c746-4b81-ae86-21c2f134ada4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:27.682219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:27.688193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 216/435] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:27 2022] GET /identity/v3/role_assignments?group.id=d31c3498b9fa46db8fe60abd91e074d9&scope.project.id=78bd72df2adb4ac4a5cc614f9d1b9167&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:27.785890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b8ba9-70f2-4f4b-8f52-85740f4ad825 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:27.809987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1fb85-32d7-48f0-aada-0d2fe5b5de77 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.812029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1fb85-32d7-48f0-aada-0d2fe5b5de77 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.812683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1fb85-32d7-48f0-aada-0d2fe5b5de77 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.814779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 217/436] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:27.821004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31dcd3-4ae7-4028-9a32-93c9c6f3d61c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.821685 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31dcd3-4ae7-4028-9a32-93c9c6f3d61c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.822023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31dcd3-4ae7-4028-9a32-93c9c6f3d61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.832601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb782-53ce-4061-930e-15ef0cb79964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:27.833686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb782-53ce-4061-930e-15ef0cb79964 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:27.834267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb782-53ce-4061-930e-15ef0cb79964 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:27.836379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb782-53ce-4061-930e-15ef0cb79964 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:27.843087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ea31dcd3-4ae7-4028-9a32-93c9c6f3d61c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:27.844378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ea31dcd3-4ae7-4028-9a32-93c9c6f3d61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:27.937448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fb782-53ce-4061-930e-15ef0cb79964 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:27.938973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fb782-53ce-4061-930e-15ef0cb79964 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:27.949159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 218/437] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:27.951432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:27.967822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31dcd3-4ae7-4028-9a32-93c9c6f3d61c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.068137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 220/438] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:28.075916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb782-53ce-4061-930e-15ef0cb79964 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.100361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31dcd3-4ae7-4028-9a32-93c9c6f3d61c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:28.138070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:28.148508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed85c-dd0c-4040-af15-5199395787e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:28.149385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed85c-dd0c-4040-af15-5199395787e2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.149666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed85c-dd0c-4040-af15-5199395787e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.149937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed85c-dd0c-4040-af15-5199395787e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.163087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ed85c-dd0c-4040-af15-5199395787e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ee5ed85c-dd0c-4040-af15-5199395787e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:28.172601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:28.173619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:28.174443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 219/439] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:28.181834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed85c-dd0c-4040-af15-5199395787e2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.200918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb782-53ce-4061-930e-15ef0cb79964 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:28.209382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:28.244150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef0bf1-b365-4319-bc8d-c2e60bb5f8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:28.245503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef0bf1-b365-4319-bc8d-c2e60bb5f8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.246013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef0bf1-b365-4319-bc8d-c2e60bb5f8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.246483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ef0bf1-b365-4319-bc8d-c2e60bb5f8f9 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.248289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef0bf1-b365-4319-bc8d-c2e60bb5f8f9 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:28.251076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ef0bf1-b365-4319-bc8d-c2e60bb5f8f9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:28.252036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 221/440] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:28.258417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef0bf1-b365-4319-bc8d-c2e60bb5f8f9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.318572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed85c-dd0c-4040-af15-5199395787e2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:28.352544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c2122-dc19-4a97-9897-6c4eabba0f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:28.353649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c2122-dc19-4a97-9897-6c4eabba0f38 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.353926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c2122-dc19-4a97-9897-6c4eabba0f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.354200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c2122-dc19-4a97-9897-6c4eabba0f38 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.354956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c2122-dc19-4a97-9897-6c4eabba0f38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:28.355269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c2122-dc19-4a97-9897-6c4eabba0f38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:28.356666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c2122-dc19-4a97-9897-6c4eabba0f38 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:28.365039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 220/441] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:28.366375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6592-4443-404c-b229-61abbf92d531 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.366700 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6592-4443-404c-b229-61abbf92d531 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.366948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e6592-4443-404c-b229-61abbf92d531 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.368336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 221/442] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:28.371086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e6592-4443-404c-b229-61abbf92d531 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.371993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ef0bf1-b365-4319-bc8d-c2e60bb5f8f9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:28.406486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa450dbf-9c66-4e88-b394-10b4af8b6d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:28.407383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa450dbf-9c66-4e88-b394-10b4af8b6d31 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.407667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa450dbf-9c66-4e88-b394-10b4af8b6d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.407945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa450dbf-9c66-4e88-b394-10b4af8b6d31 None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.409158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa450dbf-9c66-4e88-b394-10b4af8b6d31 None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:28.410657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa450dbf-9c66-4e88-b394-10b4af8b6d31 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:28.412527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 222/443] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:28.413696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4130d1-e0f7-4a93-bc33-f8d203a34ccd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.413983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4130d1-e0f7-4a93-bc33-f8d203a34ccd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.414258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4130d1-e0f7-4a93-bc33-f8d203a34ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.429076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:28.431823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7c4130d1-e0f7-4a93-bc33-f8d203a34ccd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:28.433048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7c4130d1-e0f7-4a93-bc33-f8d203a34ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:28.467843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:28.480118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e6592-4443-404c-b229-61abbf92d531 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:28.502327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 223/444] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:28.503199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4130d1-e0f7-4a93-bc33-f8d203a34ccd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.517223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04c2c3-ad01-49a4-885a-ca9452806844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:28.517912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04c2c3-ad01-49a4-885a-ca9452806844 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.518342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04c2c3-ad01-49a4-885a-ca9452806844 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.518619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04c2c3-ad01-49a4-885a-ca9452806844 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.591457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04c2c3-ad01-49a4-885a-ca9452806844 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:28.592962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04c2c3-ad01-49a4-885a-ca9452806844 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:28.602360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4130d1-e0f7-4a93-bc33-f8d203a34ccd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:28.635845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:28.636506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.636760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.637037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.637896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] RBAC: Adding query filter params (group.id=d31c3498b9fa46db8fe60abd91e074d9, role.id=27d4ad049d8f4ff4a176581dea972512, scope.project.id=fc31ecc9189941fa909edb765d79c496) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:28.638210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:28.639673 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:28.643892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:28.657211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 224/445] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/role_assignments?group.id=d31c3498b9fa46db8fe60abd91e074d9&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=27d4ad049d8f4ff4a176581dea972512 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:28.657211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.685403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:28.703732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 222/446] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:09:28.710280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04c2c3-ad01-49a4-885a-ca9452806844 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.764307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d900cb-7c50-4628-ae30-cef00d825c5f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:28.801594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617043da-95b7-4fc3-b970-ceb6626be991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:28.802434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617043da-95b7-4fc3-b970-ceb6626be991 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.802745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617043da-95b7-4fc3-b970-ceb6626be991 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.803021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617043da-95b7-4fc3-b970-ceb6626be991 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.816047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617043da-95b7-4fc3-b970-ceb6626be991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-617043da-95b7-4fc3-b970-ceb6626be991 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:28.824742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:28.825701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:28.826783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 225/447] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:28.831919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617043da-95b7-4fc3-b970-ceb6626be991 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.832413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04c2c3-ad01-49a4-885a-ca9452806844 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:28.875324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4dccf-1f46-433e-b8c5-7ac6d3b837a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:28.876219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4dccf-1f46-433e-b8c5-7ac6d3b837a0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:28.876496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4dccf-1f46-433e-b8c5-7ac6d3b837a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:28.876770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4dccf-1f46-433e-b8c5-7ac6d3b837a0 None admin] PATH_INFO: `/v3/projects/546ad1026cd44cbea571239c12f41a3f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:28.879064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4dccf-1f46-433e-b8c5-7ac6d3b837a0 None admin] RBAC: Authorizing `identity:get_project(project_id=546ad1026cd44cbea571239c12f41a3f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:28.880541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef4dccf-1f46-433e-b8c5-7ac6d3b837a0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:28.882680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 223/448] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/projects/546ad1026cd44cbea571239c12f41a3f => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:28.888475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4dccf-1f46-433e-b8c5-7ac6d3b837a0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:28.914905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:28.947835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:28.955853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617043da-95b7-4fc3-b970-ceb6626be991 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:29.001307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:29.001307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:29.001307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:29.001623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:29.002281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:29.002743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:29.004119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:29.008514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef4dccf-1f46-433e-b8c5-7ac6d3b837a0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:29.013829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 226/449] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:29.019502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:29.048720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f39028-ddb4-44f1-836f-5dfe423db7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:29.049658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f39028-ddb4-44f1-836f-5dfe423db7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:29.049938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f39028-ddb4-44f1-836f-5dfe423db7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:29.050210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f39028-ddb4-44f1-836f-5dfe423db7b3 None admin] PATH_INFO: `/v3/groups/65554ceb43da47faa302f2e55fada658` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:29.051390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f39028-ddb4-44f1-836f-5dfe423db7b3 None admin] RBAC: Authorizing `identity:get_group(group_id=65554ceb43da47faa302f2e55fada658)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:29.052964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f39028-ddb4-44f1-836f-5dfe423db7b3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:29.055557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 224/450] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:28 2022] GET /identity/v3/groups/65554ceb43da47faa302f2e55fada658 => generated 229 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:29.061281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f39028-ddb4-44f1-836f-5dfe423db7b3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:29.126005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6fa8e1-c475-4c98-943c-6b5cb9cbe2fd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:29.149017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:29.158984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e97b4b-7fd0-4059-9f2b-8433e757a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:29.159651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e97b4b-7fd0-4059-9f2b-8433e757a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:29.159926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e97b4b-7fd0-4059-9f2b-8433e757a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:29.160200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e97b4b-7fd0-4059-9f2b-8433e757a1b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:29.161325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f39028-ddb4-44f1-836f-5dfe423db7b3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:29.187196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:29.195722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:29.196418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:29.196702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:29.196986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:29.197851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] RBAC: Adding query filter params (group.id=65554ceb43da47faa302f2e55fada658, role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=546ad1026cd44cbea571239c12f41a3f) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:29.198168 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:29.199620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:29.205934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 225/451] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:29 2022] GET /identity/v3/role_assignments?group.id=65554ceb43da47faa302f2e55fada658&scope.project.id=546ad1026cd44cbea571239c12f41a3f&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:29.232964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e97b4b-7fd0-4059-9f2b-8433e757a1b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:29.234496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e97b4b-7fd0-4059-9f2b-8433e757a1b0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:29.345142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 227/452] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:29.351464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:29.460428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b51d5-a9d1-4526-9b62-f58b1beef947 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:29.495334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed503-38a5-46ee-992f-0151d48977b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:29.496225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed503-38a5-46ee-992f-0151d48977b9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:29.496505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed503-38a5-46ee-992f-0151d48977b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:29.496777 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed503-38a5-46ee-992f-0151d48977b9 None admin] PATH_INFO: `/v3/projects/fc31ecc9189941fa909edb765d79c496` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:29.498041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed503-38a5-46ee-992f-0151d48977b9 None admin] RBAC: Authorizing `identity:get_project(project_id=fc31ecc9189941fa909edb765d79c496)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:29.499506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed503-38a5-46ee-992f-0151d48977b9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:29.501213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 226/453] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:09:29 2022] GET /identity/v3/projects/fc31ecc9189941fa909edb765d79c496 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:29.507230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e97b4b-7fd0-4059-9f2b-8433e757a1b0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:29.566747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:29.616406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e97b4b-7fd0-4059-9f2b-8433e757a1b0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:29.653985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f9e24-5db9-4b50-b364-1e159b03451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:29.654854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f9e24-5db9-4b50-b364-1e159b03451e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:29.655132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f9e24-5db9-4b50-b364-1e159b03451e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:29.655410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f9e24-5db9-4b50-b364-1e159b03451e None admin] PATH_INFO: `/v3/groups/d31c3498b9fa46db8fe60abd91e074d9` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:29.656703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f9e24-5db9-4b50-b364-1e159b03451e None admin] RBAC: Authorizing `identity:get_group(group_id=d31c3498b9fa46db8fe60abd91e074d9)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:29.658190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f9e24-5db9-4b50-b364-1e159b03451e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:29.659844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 228/454] 10.222.0.66 () {56 vars in 1180 bytes} [Thu Jun 16 17:09:29 2022] GET /identity/v3/groups/d31c3498b9fa46db8fe60abd91e074d9 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:29.668634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed503-38a5-46ee-992f-0151d48977b9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:29.692303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:29.772973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed503-38a5-46ee-992f-0151d48977b9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:29.808180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef24f3e5-80dd-48be-872e-92f4a3f962d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:29.808836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f3e5-80dd-48be-872e-92f4a3f962d4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:29.809115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f3e5-80dd-48be-872e-92f4a3f962d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:29.809416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef24f3e5-80dd-48be-872e-92f4a3f962d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:29.810255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24f3e5-80dd-48be-872e-92f4a3f962d4 None admin] RBAC: Adding query filter params (group.id=d31c3498b9fa46db8fe60abd91e074d9, role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=fc31ecc9189941fa909edb765d79c496) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:29.810570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24f3e5-80dd-48be-872e-92f4a3f962d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:29.812037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef24f3e5-80dd-48be-872e-92f4a3f962d4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:29.818138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 227/455] 10.222.0.66 () {56 vars in 1376 bytes} [Thu Jun 16 17:09:29 2022] GET /identity/v3/role_assignments?group.id=d31c3498b9fa46db8fe60abd91e074d9&scope.project.id=fc31ecc9189941fa909edb765d79c496&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:32.559319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b035c-e51c-425c-b962-e0bf70208fd4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:32.560277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b035c-e51c-425c-b962-e0bf70208fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:32.560771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b035c-e51c-425c-b962-e0bf70208fd4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:32.562664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 229/456] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:32.568181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c72884-2ca2-4236-b8f9-1d83824721b6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:32.569148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c72884-2ca2-4236-b8f9-1d83824721b6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:32.569656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c72884-2ca2-4236-b8f9-1d83824721b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:32.593392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e7c72884-2ca2-4236-b8f9-1d83824721b6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:32.594987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e7c72884-2ca2-4236-b8f9-1d83824721b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:32.687145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 228/457] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:32.726777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b035c-e51c-425c-b962-e0bf70208fd4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:32.874130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b035c-e51c-425c-b962-e0bf70208fd4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:32.915382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:32.923181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d13e-5e05-4f40-8614-0c7b4907cb82 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:32.923949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d13e-5e05-4f40-8614-0c7b4907cb82 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:32.924532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9d13e-5e05-4f40-8614-0c7b4907cb82 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:32.926851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 229/458] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:32.931821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d62706c-fc1a-4d53-b20e-a981246a00f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:32.932661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62706c-fc1a-4d53-b20e-a981246a00f4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:32.932938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62706c-fc1a-4d53-b20e-a981246a00f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:32.933225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d62706c-fc1a-4d53-b20e-a981246a00f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:32.933482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b3b84-7b6f-427e-a65e-cb0a31bb7751 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:32.934149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b3b84-7b6f-427e-a65e-cb0a31bb7751 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:32.934616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b3b84-7b6f-427e-a65e-cb0a31bb7751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:32.944360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62706c-fc1a-4d53-b20e-a981246a00f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:32.945889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d62706c-fc1a-4d53-b20e-a981246a00f4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:32.962768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c24b3b84-7b6f-427e-a65e-cb0a31bb7751 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:32.964741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c24b3b84-7b6f-427e-a65e-cb0a31bb7751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0d62706c-fc1a-4d53-b20e-a981246a00f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:32.966126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:32.967028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:32.967902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:32.967902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:32.967902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:32.967902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:32.967902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 230/459] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:32.968933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd56f-9e18-4e0b-b7e5-247377637655 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:32.969237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd56f-9e18-4e0b-b7e5-247377637655 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:32.969530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fd56f-9e18-4e0b-b7e5-247377637655 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:32.972263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 231/460] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:32.977024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd56f-9e18-4e0b-b7e5-247377637655 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:32.980079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:33.057199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 230/461] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:33.057199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d21ccc-e05e-48fd-803b-31a1858e3b18 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.057199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d21ccc-e05e-48fd-803b-31a1858e3b18 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.057199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d21ccc-e05e-48fd-803b-31a1858e3b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.087583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-21d21ccc-e05e-48fd-803b-31a1858e3b18 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:33.089193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21d21ccc-e05e-48fd-803b-31a1858e3b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:33.113833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fd56f-9e18-4e0b-b7e5-247377637655 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:33.151038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:33.151708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.152152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.152652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.153620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:33.154126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:33.155930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:33.160845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 231/462] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:33.162122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d21ccc-e05e-48fd-803b-31a1858e3b18 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:33.176612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 232/463] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:33.182260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:33.239817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:33.261624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:33.270109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d21ccc-e05e-48fd-803b-31a1858e3b18 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:33.307722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b23d7e9-8de3-47d1-9ab6-fa39e584545f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:33.314197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:33.315340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.315761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.316234 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.328984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:33.330739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:33.346234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437bd91-1500-4431-ad3e-a93a61ca92dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:33.347249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437bd91-1500-4431-ad3e-a93a61ca92dc None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.347735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437bd91-1500-4431-ad3e-a93a61ca92dc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.348206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437bd91-1500-4431-ad3e-a93a61ca92dc None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.349139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437bd91-1500-4431-ad3e-a93a61ca92dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:33.351064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437bd91-1500-4431-ad3e-a93a61ca92dc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:33.351857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:33.352798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:33.353727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:33.353727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:33.353727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:33.353727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:33.353727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 232/464] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:33.354005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:33.375758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 233/465] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:09:33 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:09:33.377396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58d7bf-2cdb-448a-b442-2ef3b944b6ad None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.377692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58d7bf-2cdb-448a-b442-2ef3b944b6ad None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.377973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58d7bf-2cdb-448a-b442-2ef3b944b6ad None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.379251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 234/466] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:33.380222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58d7bf-2cdb-448a-b442-2ef3b944b6ad None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:33.504380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee64b05-733d-4432-a70d-e768715e6caf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:33.537420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:33.547919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:33.549478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8aa60-87ad-4368-b048-f514369b1d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:33.550945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8aa60-87ad-4368-b048-f514369b1d79 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.551381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8aa60-87ad-4368-b048-f514369b1d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.551806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8aa60-87ad-4368-b048-f514369b1d79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.558853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58d7bf-2cdb-448a-b442-2ef3b944b6ad None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:33.568115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8aa60-87ad-4368-b048-f514369b1d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:33.569969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8aa60-87ad-4368-b048-f514369b1d79 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d2e8aa60-87ad-4368-b048-f514369b1d79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:33.591074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:33.592003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:33.592894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:33.592894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:33.592894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:33.592894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:33.594606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 233/467] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:33.596013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed32f-2462-4e5b-965b-4586e2042f2e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.596431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed32f-2462-4e5b-965b-4586e2042f2e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.596852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ed32f-2462-4e5b-965b-4586e2042f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.598136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:33.598916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.599260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.599745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.600555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:33.601222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:33.602603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:33.615636 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-275ed32f-2462-4e5b-965b-4586e2042f2e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:33.617074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-275ed32f-2462-4e5b-965b-4586e2042f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:33.619610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 235/468] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:33.620512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:33.694882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 234/469] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:33.695726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ed32f-2462-4e5b-965b-4586e2042f2e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:33.756085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f82f2-8cb3-43eb-93e9-9bae9832ecf3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:33.799058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501df343-3a43-4b6f-8975-315596412122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:33.800279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df343-3a43-4b6f-8975-315596412122 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.800783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df343-3a43-4b6f-8975-315596412122 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.801285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501df343-3a43-4b6f-8975-315596412122 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.802383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501df343-3a43-4b6f-8975-315596412122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:33.803011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501df343-3a43-4b6f-8975-315596412122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:33.804586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501df343-3a43-4b6f-8975-315596412122 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:33.804923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ed32f-2462-4e5b-965b-4586e2042f2e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:33.817503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 236/470] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:33.822158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501df343-3a43-4b6f-8975-315596412122 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:33.822295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:33.843982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:33.850981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69403c-cafa-4030-ab1d-90e5b00d01ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:33.851757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69403c-cafa-4030-ab1d-90e5b00d01ad None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:33.852046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69403c-cafa-4030-ab1d-90e5b00d01ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:33.852328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe69403c-cafa-4030-ab1d-90e5b00d01ad None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:33.853117 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69403c-cafa-4030-ab1d-90e5b00d01ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:33.855333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe69403c-cafa-4030-ab1d-90e5b00d01ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:33.885283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 235/471] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:09:33 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:09:33.887075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69403c-cafa-4030-ab1d-90e5b00d01ad None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:34.017519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501df343-3a43-4b6f-8975-315596412122 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:34.063713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe69403c-cafa-4030-ab1d-90e5b00d01ad None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:34.069123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cc89e-4a89-4441-b927-93ece691887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:34.070270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc89e-4a89-4441-b927-93ece691887a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.070558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc89e-4a89-4441-b927-93ece691887a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.070840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cc89e-4a89-4441-b927-93ece691887a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.090087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cc89e-4a89-4441-b927-93ece691887a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:34.092075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cc89e-4a89-4441-b927-93ece691887a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:34.111705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f608-50b6-4765-9df9-4fc3c3862953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:34.112406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f608-50b6-4765-9df9-4fc3c3862953 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.112698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f608-50b6-4765-9df9-4fc3c3862953 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b49cc89e-4a89-4441-b927-93ece691887a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:34.112802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:34.113704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:34.114999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:34.114999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:34.114999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:34.114999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:34.115271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e4f608-50b6-4765-9df9-4fc3c3862953 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.115271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4f608-50b6-4765-9df9-4fc3c3862953 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:34.115391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 237/472] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:34.115466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e4f608-50b6-4765-9df9-4fc3c3862953 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:34.115730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac56eb-f711-4f19-a76c-34b45a5c3c5c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.116976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac56eb-f711-4f19-a76c-34b45a5c3c5c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.116976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac56eb-f711-4f19-a76c-34b45a5c3c5c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.118932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 238/473] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:34.127552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5c0a7-2ed5-49fa-ae57-c20621c1835f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.128008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5c0a7-2ed5-49fa-ae57-c20621c1835f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.128365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5c0a7-2ed5-49fa-ae57-c20621c1835f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.141580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 236/474] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:09:33 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:09:34.142437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f608-50b6-4765-9df9-4fc3c3862953 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:34.152831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-89e5c0a7-2ed5-49fa-ae57-c20621c1835f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:34.154271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-89e5c0a7-2ed5-49fa-ae57-c20621c1835f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:34.175646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:34.290446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:34.293671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e4f608-50b6-4765-9df9-4fc3c3862953 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:34.318133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 239/475] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:34.336133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:34.336852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.337135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.337547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.338285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:34.338594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:34.340037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:34.360229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 237/476] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:34.365502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5c0a7-2ed5-49fa-ae57-c20621c1835f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:34.365879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:34.458490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:34.476493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5c0a7-2ed5-49fa-ae57-c20621c1835f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:34.481458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1af48-9d7d-4602-981a-07d43e7aba61 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:34.514424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:34.515121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.515402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.515674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.516442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:34.517968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:34.520820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825d792-f167-4c2e-83c1-68ee4f829c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:34.521761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825d792-f167-4c2e-83c1-68ee4f829c36 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.522051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825d792-f167-4c2e-83c1-68ee4f829c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.522436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825d792-f167-4c2e-83c1-68ee4f829c36 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.523165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825d792-f167-4c2e-83c1-68ee4f829c36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:34.524930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825d792-f167-4c2e-83c1-68ee4f829c36 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:34.525789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.sql.core [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:09:34.525789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 17:09:34.525789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [parameters: {'id': 'f7e7f1a56ffa42a89b8caf9e9e6aec07', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 17:09:34.525789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80239) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result.read() Jun 16 17:09:34.545325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:09:34.546251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application yield resource Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise self.value Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application session.commit() Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:09:34.547194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:09:34.548044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:09:34.548947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result.read() Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:09:34.549822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [parameters: {'id': 'f7e7f1a56ffa42a89b8caf9e9e6aec07', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:34.550638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 17:09:34.551471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:34.552329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 238/477] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:09:34 2022] POST /identity/v3/users => generated 249 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:09:34.554912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 240/478] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Jun 16 17:09:34 2022] POST /identity/v3/roles => generated 144 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:34.560858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825d792-f167-4c2e-83c1-68ee4f829c36 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:34.674604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825d792-f167-4c2e-83c1-68ee4f829c36 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:34.712210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f7a0d-4dd5-44d0-9afe-1bd8187d0570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:34.713138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f7a0d-4dd5-44d0-9afe-1bd8187d0570 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.713459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f7a0d-4dd5-44d0-9afe-1bd8187d0570 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.713738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f7a0d-4dd5-44d0-9afe-1bd8187d0570 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.717043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:34.731274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f7a0d-4dd5-44d0-9afe-1bd8187d0570 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cf3f7a0d-4dd5-44d0-9afe-1bd8187d0570 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:34.748674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:34.750799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:34.752626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 239/479] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:09:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:34.756479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:34.832083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:34.881474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5bbc7f-d4e3-45df-a97f-f6c2706c85bc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:34.916353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e606c-f622-4f1b-8804-9645dd73743a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:34.917453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e606c-f622-4f1b-8804-9645dd73743a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:34.917877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e606c-f622-4f1b-8804-9645dd73743a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:34.918323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e606c-f622-4f1b-8804-9645dd73743a None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:34.919263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e606c-f622-4f1b-8804-9645dd73743a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:34.919729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e606c-f622-4f1b-8804-9645dd73743a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:34.921260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e606c-f622-4f1b-8804-9645dd73743a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:34.930735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 241/480] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:36.172600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191a67-1872-4ce1-927d-2b2ef9358d84 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.173202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191a67-1872-4ce1-927d-2b2ef9358d84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.173202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191a67-1872-4ce1-927d-2b2ef9358d84 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.175042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 240/481] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:36.181093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829a565-f085-4147-839e-8ee22d232e1c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.181633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829a565-f085-4147-839e-8ee22d232e1c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.181738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829a565-f085-4147-839e-8ee22d232e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.201875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e829a565-f085-4147-839e-8ee22d232e1c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:36.203110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e829a565-f085-4147-839e-8ee22d232e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:36.277247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 242/482] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:36.300924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191a67-1872-4ce1-927d-2b2ef9358d84 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:36.442006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191a67-1872-4ce1-927d-2b2ef9358d84 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:36.455698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2f4b-3b78-4386-bd59-383ec70eb659 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.456005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2f4b-3b78-4386-bd59-383ec70eb659 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.456253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef2f4b-3b78-4386-bd59-383ec70eb659 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.457698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 243/483] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:09:36.466771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9505-8c0f-4935-93eb-e6ac541179d7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.467059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9505-8c0f-4935-93eb-e6ac541179d7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.467335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca9505-8c0f-4935-93eb-e6ac541179d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.494351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c8ca9505-8c0f-4935-93eb-e6ac541179d7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:36.495579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c8ca9505-8c0f-4935-93eb-e6ac541179d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:36.515448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5c914-b6ed-430e-8023-1d480470a238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:36.516275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5c914-b6ed-430e-8023-1d480470a238 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.516555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5c914-b6ed-430e-8023-1d480470a238 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.516831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5c914-b6ed-430e-8023-1d480470a238 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.536874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5c914-b6ed-430e-8023-1d480470a238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-39a5c914-b6ed-430e-8023-1d480470a238 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:36.544984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:36.545913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:36.546761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:36.547897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 241/484] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:36.556017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5c914-b6ed-430e-8023-1d480470a238 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:36.581568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:36.621296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 244/485] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:36.660714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:36.667573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca9505-8c0f-4935-93eb-e6ac541179d7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:36.755856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5c914-b6ed-430e-8023-1d480470a238 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:36.796945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca9505-8c0f-4935-93eb-e6ac541179d7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:36.810325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f688f-da47-4b14-bfc0-da4beaf7b2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:36.811095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f688f-da47-4b14-bfc0-da4beaf7b2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.811381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f688f-da47-4b14-bfc0-da4beaf7b2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.811661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f688f-da47-4b14-bfc0-da4beaf7b2d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.812477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f688f-da47-4b14-bfc0-da4beaf7b2d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:36.812799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f688f-da47-4b14-bfc0-da4beaf7b2d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:36.814494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f688f-da47-4b14-bfc0-da4beaf7b2d4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:36.826938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 242/486] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:36.828433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db94263-dc32-4869-876d-a0fe0b15c332 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.828730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db94263-dc32-4869-876d-a0fe0b15c332 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.829009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db94263-dc32-4869-876d-a0fe0b15c332 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.830604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 243/487] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:36.834168 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db94263-dc32-4869-876d-a0fe0b15c332 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:36.848732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ccd9c-3da7-412a-bca9-7db4bb7ecfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:36.849657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ccd9c-3da7-412a-bca9-7db4bb7ecfbe None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.849942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ccd9c-3da7-412a-bca9-7db4bb7ecfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.850267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ccd9c-3da7-412a-bca9-7db4bb7ecfbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.861414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ccd9c-3da7-412a-bca9-7db4bb7ecfbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e70ccd9c-3da7-412a-bca9-7db4bb7ecfbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:36.869615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:36.870557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 245/488] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:36.876693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17efe1-eef4-4723-b64f-a4c27dba815a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:36.877700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17efe1-eef4-4723-b64f-a4c27dba815a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:36.877700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17efe1-eef4-4723-b64f-a4c27dba815a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:36.895099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8e17efe1-eef4-4723-b64f-a4c27dba815a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:36.896479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8e17efe1-eef4-4723-b64f-a4c27dba815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:36.934365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:36.960815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db94263-dc32-4869-876d-a0fe0b15c332 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:36.969107 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 246/489] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:36.969952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17efe1-eef4-4723-b64f-a4c27dba815a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:36.980916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:37.001344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.003187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.003684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.004161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.018407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.020417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:37.040120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:37.041041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:37.042274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:37.042274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:37.042274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:37.042274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:37.048882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 244/490] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:37.050131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.075538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17efe1-eef4-4723-b64f-a4c27dba815a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.113328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.114728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.115082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.115381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.116156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:37.116685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.119375 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.131112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 247/491] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:37.132340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.173933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13821fb0-f105-4605-8b45-db4ccef05b94 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.174765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:37.219166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f4a4-7efc-405b-ad3a-630f3593ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.220347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f4a4-7efc-405b-ad3a-630f3593ebae None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.220632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f4a4-7efc-405b-ad3a-630f3593ebae None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.220910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f4a4-7efc-405b-ad3a-630f3593ebae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.235822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25f4a4-7efc-405b-ad3a-630f3593ebae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-af25f4a4-7efc-405b-ad3a-630f3593ebae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:37.246180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:37.247137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:37.248367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 245/492] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:37.249321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f4a4-7efc-405b-ad3a-630f3593ebae None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.262509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad4f0f2-cd99-4778-8099-9b2bb7817697 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.297712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:37.298742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.299431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.299959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.299959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.300749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:37.301069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.302760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.321098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 248/493] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:37.321098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.372306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f4a4-7efc-405b-ad3a-630f3593ebae None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.410874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.412133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.412829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.413365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.425654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:37.426883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.428830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.434323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073838e4-0de3-4f8c-9cc0-2d89d12a749c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:37.448610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:37.449598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:37.450486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:37.450486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:37.450486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:37.450486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:37.450486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 246/494] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:37.451212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.475464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87096c0-0c68-4fc4-9bbb-931ad6bdf91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.476479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87096c0-0c68-4fc4-9bbb-931ad6bdf91e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.476787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87096c0-0c68-4fc4-9bbb-931ad6bdf91e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.477044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87096c0-0c68-4fc4-9bbb-931ad6bdf91e None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.477904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87096c0-0c68-4fc4-9bbb-931ad6bdf91e None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:37.478218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87096c0-0c68-4fc4-9bbb-931ad6bdf91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.479633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87096c0-0c68-4fc4-9bbb-931ad6bdf91e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.493029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 249/495] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:37.494592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8ce84-3c0b-473f-85ff-d4868d0f2abf None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.494886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8ce84-3c0b-473f-85ff-d4868d0f2abf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.495172 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8ce84-3c0b-473f-85ff-d4868d0f2abf None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.496537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 250/496] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:37.497359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8ce84-3c0b-473f-85ff-d4868d0f2abf None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.558441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:37.578806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eacbe3-5f40-4423-b032-b00f74a34e3a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.608831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8ce84-3c0b-473f-85ff-d4868d0f2abf None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.616069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.617166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.617616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.618037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.626985 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.628671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.643267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac3533-36b1-48e2-a3d1-d47dde119741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.643968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3533-36b1-48e2-a3d1-d47dde119741 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.644250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3533-36b1-48e2-a3d1-d47dde119741 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.644529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac3533-36b1-48e2-a3d1-d47dde119741 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.645441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac3533-36b1-48e2-a3d1-d47dde119741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:37.645758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac3533-36b1-48e2-a3d1-d47dde119741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:37.646532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:37.647629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:09:37.648771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:37.649452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac3533-36b1-48e2-a3d1-d47dde119741 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.651315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 247/497] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:37.652258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.662420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 251/498] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:37.663857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745867d-9ada-4b9f-8069-b465977b1fd8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.664409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745867d-9ada-4b9f-8069-b465977b1fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.665338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745867d-9ada-4b9f-8069-b465977b1fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.686188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e745867d-9ada-4b9f-8069-b465977b1fd8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:37.687462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e745867d-9ada-4b9f-8069-b465977b1fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:37.703028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:37.769358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 252/499] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:37.769358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745867d-9ada-4b9f-8069-b465977b1fd8 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.773484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3b9b1-54cf-4673-8bc9-692e65ab514b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.815266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.816011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.816295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.816573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.828333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.829934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.832042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:37.850457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:37.851424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:37.852291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:37.852291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:37.852291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:37.852291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:37.852291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 248/500] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:37.852563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.895652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745867d-9ada-4b9f-8069-b465977b1fd8 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.931293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:37.931984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:37.932252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:37.932540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:37.933401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:37.933714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:37.935160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:37.969531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461af6e8-0a73-4970-ac2d-868434bc24be None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:37.972491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 253/501] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:37.973348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:37.978219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.007046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.007046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.007342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.007643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.015948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:38.017473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:38.032498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:38.033595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 249/502] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:38.034500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.079005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3facd2-4635-4009-abfa-12649e06254d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:38.097136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.115234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe946be1-fe15-44f6-bbab-0a74a124aa0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.116219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe946be1-fe15-44f6-bbab-0a74a124aa0d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.116509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe946be1-fe15-44f6-bbab-0a74a124aa0d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.116788 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe946be1-fe15-44f6-bbab-0a74a124aa0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.127444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe946be1-fe15-44f6-bbab-0a74a124aa0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-fe946be1-fe15-44f6-bbab-0a74a124aa0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:38.135363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:38.136434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:38.137324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 254/503] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:38.138010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d7823-010a-4939-a97b-67ea50656817 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.138195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d7823-010a-4939-a97b-67ea50656817 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.138481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d7823-010a-4939-a97b-67ea50656817 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.139739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 255/504] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:38.140431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d7823-010a-4939-a97b-67ea50656817 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.150850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff0a2c-fca0-4007-a717-e9ea5950d1d0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:38.191042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.191743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.191998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.192306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.193053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:38.193968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:38.195419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:38.208121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 250/505] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:38.210050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.217476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.257394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d7823-010a-4939-a97b-67ea50656817 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:38.292273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23443fbc-cf2e-445a-b416-f9773a8a2414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.292957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23443fbc-cf2e-445a-b416-f9773a8a2414 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.293213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23443fbc-cf2e-445a-b416-f9773a8a2414 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.293509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23443fbc-cf2e-445a-b416-f9773a8a2414 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.319319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b46626-9134-420f-954d-673135d3ab51 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:38.335861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.358017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.358713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.358989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.359260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.360098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:38.360409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:38.361898 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:38.362599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23443fbc-cf2e-445a-b416-f9773a8a2414 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:38.364088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23443fbc-cf2e-445a-b416-f9773a8a2414 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:38.399474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 251/506] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:38.400270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.429144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.469567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 256/507] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:38.471168 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206970-09f1-4062-ae16-8ae5c5fa177d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.471453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206970-09f1-4062-ae16-8ae5c5fa177d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.471727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13206970-09f1-4062-ae16-8ae5c5fa177d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.491358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-13206970-09f1-4062-ae16-8ae5c5fa177d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:38.492581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-13206970-09f1-4062-ae16-8ae5c5fa177d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:38.511136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8636f-633a-498b-adac-cce5a71e04b5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:38.544914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.545614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.545890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.546162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.546910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:38.547230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:38.548862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:38.557023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 252/508] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:38.557869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.558313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 257/509] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:38.559197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13206970-09f1-4062-ae16-8ae5c5fa177d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.610504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.651413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.659073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13206970-09f1-4062-ae16-8ae5c5fa177d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:38.676756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a4d25-dbb5-4142-b909-0f96625419d6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:38.692699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c946f09-403e-43db-8931-f244f1e0ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.693759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c946f09-403e-43db-8931-f244f1e0ab2c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.694006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c946f09-403e-43db-8931-f244f1e0ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.694279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c946f09-403e-43db-8931-f244f1e0ab2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.713577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.714388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.714672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.714946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] PATH_INFO: `/v3/users/glance` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.723159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:38.725309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:38.740133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:38.741150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:38.742128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 253/510] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:38.742795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.771085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c946f09-403e-43db-8931-f244f1e0ab2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:38.772758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c946f09-403e-43db-8931-f244f1e0ab2c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:38.845813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.854167 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f07878-82ab-4a43-9e07-fea0f5b2bdff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:38.868320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:38.876537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 258/511] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:38.877644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c946f09-403e-43db-8931-f244f1e0ab2c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.891097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:38.891936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:38.892229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:38.892510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:38.904141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:38.905680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:38.923081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:38.923968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:38.925875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:38.925875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:38.925875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:38.925875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:38.925875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 254/512] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:38.925875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:38.981994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c946f09-403e-43db-8931-f244f1e0ab2c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.019027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.019906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.020325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.020603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.032188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.033746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.034235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754aec53-7b68-46c1-ba50-aad18c7f5ac8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:39.051444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:39.052449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:39.054217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:39.054217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:39.054217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:39.054217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:39.054217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 259/513] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:39.054217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.073844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.074673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.074984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.075265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.086869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.088341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.094815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:39.105827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:39.106333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:39.107311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:39.108239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:39.108239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:39.108239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:39.108239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:39.108239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 255/514] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:39.108521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.167724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6469c3a-c923-47ad-ac0a-09a77e01ac93 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.201778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.202469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.203027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.203288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.204223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:39.204639 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.206156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.209369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e99500-7ef2-4cb2-bea9-0f30dce35776 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.226017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 260/515] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:39.226913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.247533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.248357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.248630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.248908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.260430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.261993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:39.280041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:39.280944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:39.282031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:39.282031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:39.282031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:39.282031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:39.282031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 256/516] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:39.282031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.333257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:39.339053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568f944-0e91-4343-99d8-e96dce06cd42 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.341493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:39.375826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.376508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.376777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.377052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.377840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:39.378148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.379533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.386102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b2638e-e32e-47bd-b1c1-c29b9dd6a841 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.391653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 261/517] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:39.392500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.423169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.423801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.424081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.424352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.425086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:39.425433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.426873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.439056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 257/518] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:39.439873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.495113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12bb14e-b399-424d-b9a8-44ba6c6c20ed None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.530411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.531098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.531381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.531661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.532408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:39.532742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.534230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.545424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dace4f8-f839-465c-a9cc-47ef51de4d2a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.547303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 262/519] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:39.547985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.557263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:39.577695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:39.587567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2365949-8744-49ed-aa87-bd4ea2ab3802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.588621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2365949-8744-49ed-aa87-bd4ea2ab3802 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.588988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2365949-8744-49ed-aa87-bd4ea2ab3802 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.589300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2365949-8744-49ed-aa87-bd4ea2ab3802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.657191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e3159-1501-46ba-8edd-f4b2659a9f38 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.663601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2365949-8744-49ed-aa87-bd4ea2ab3802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.665126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2365949-8744-49ed-aa87-bd4ea2ab3802 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.691571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.692246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.692530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.692806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.693670 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:39.693974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.695357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.706634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 263/520] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:39.707546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.766517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 258/521] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:39.767363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2365949-8744-49ed-aa87-bd4ea2ab3802 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.779664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:39.801287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:39.820553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293ab50-756e-4153-9d92-da1113660f4e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.854816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.856585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.856865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.857133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.864292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.866196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.872325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2365949-8744-49ed-aa87-bd4ea2ab3802 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:39.880752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:39.881981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:39.882948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 264/522] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:39.883430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.913506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:39.914347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:39.914637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:39.914920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:39.922603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:39.924100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:39.938409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:39.939499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 259/523] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:39.940435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:39.985242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14867407-5751-4247-ae8c-75cdaac0aa01 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.012034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:40.022993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a7fe7-629d-4800-8c4b-3527b22ba3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.023687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a7fe7-629d-4800-8c4b-3527b22ba3fa None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.023990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a7fe7-629d-4800-8c4b-3527b22ba3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.024257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08a7fe7-629d-4800-8c4b-3527b22ba3fa None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.024960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a7fe7-629d-4800-8c4b-3527b22ba3fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.026491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08a7fe7-629d-4800-8c4b-3527b22ba3fa None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.043136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:40.045798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071b3d9-b5c2-44a5-8535-6d37f8b6065e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.053472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 265/524] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:09:39 2022] POST /identity/v3/users => generated 247 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:09:40.054990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a7fe7-629d-4800-8c4b-3527b22ba3fa None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.088456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.089368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.089660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.089967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.098299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.099929 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:40.118209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:40.119696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:40.120782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:40.120782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:40.120782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:40.120782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:40.120782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:40.120782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:40.120782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:40.120782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:40.124790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 260/525] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:40.125680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.179334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08a7fe7-629d-4800-8c4b-3527b22ba3fa None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.221309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.222134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.222412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.222694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.235486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.236960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.242253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa887fa-cb25-43b8-8fca-521b1077622b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:40.256493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:40.257545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:40.258444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:40.258444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:40.258444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:40.258444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:40.259171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 266/526] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:40.259639 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.284080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:40.291786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.292480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.293194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.293194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.293868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:40.294336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.295651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.314946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 261/527] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:40.315635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.330865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:40.386129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a22c9a-288b-4315-9066-22807a4079f7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.423924 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.424630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.424908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.425207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.426077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:40.426386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.428199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.432157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fbdad-3641-410d-84cb-17f85a8d764c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.447280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 267/528] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:40.448169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.478251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.478962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.479246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.479529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.480752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:40.481066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.482612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.498428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 262/529] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:40.499311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.553087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:40.567291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef0252-ba52-4cad-92a0-544e559ddba3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.579528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:40.609967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.610165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc8fd1-92a4-4e79-bf59-a4046bd1d281 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.610640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.610924 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.611203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.612228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:40.613439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.614983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.636201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 268/530] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:40.636201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.653612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.654292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.654574 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.654850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.655716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=e3241452a88a40da9c9b8fa79c4fc9c0) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:40.656026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.657523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.664576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 263/531] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/role_assignments?user.id=e3241452a88a40da9c9b8fa79c4fc9c0&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:40.665717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.760466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367d3c9-6e39-467e-ab08-185fce425ab2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.786520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba2914-f373-44d5-b424-2f230dd6a33b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:40.803692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:40.806415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeff3ec-2922-4409-ae3a-f9ba078f3a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.807563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff3ec-2922-4409-ae3a-f9ba078f3a7d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.808194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff3ec-2922-4409-ae3a-f9ba078f3a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.808742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff3ec-2922-4409-ae3a-f9ba078f3a7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.827252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:40.827942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:40.828227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:40.828508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:40.829532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=d6179dc2bd1d4ff2a04a8362d63e7b7c) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:40.829759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.831387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.845329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 264/532] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/role_assignments?user.id=d6179dc2bd1d4ff2a04a8362d63e7b7c&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:40.845329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:40.864646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:40.899791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeff3ec-2922-4409-ae3a-f9ba078f3a7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:40.901581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeff3ec-2922-4409-ae3a-f9ba078f3a7d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:40.963815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dffc2-44bd-4283-ab60-37c3a61e3b4c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:41.002109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:41.003059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:41.003347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:41.003629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:41.012348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:41.013862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:41.017384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 269/533] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:41.018707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeff3ec-2922-4409-ae3a-f9ba078f3a7d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:41.031860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:41.032873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:41.033794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 265/534] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:41.039687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:41.083850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:41.108364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:41.144051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeff3ec-2922-4409-ae3a-f9ba078f3a7d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:41.149559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e922da-368a-4f32-96a0-77d863b9f099 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:41.184674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:41.186028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:41.186464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:41.186879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:41.189537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:41.190483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:41.190694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:41.191071 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:41.192012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:41.192453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:41.194120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:41.199808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:41.201545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:41.212045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 266/535] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:41.215628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:41.231611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:41.232742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:41.233835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:41.233835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:41.233835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:41.233835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:41.241199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 270/536] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:41.242988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:41.401507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:41.406506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98a79f-8444-4402-8d0a-fcbb12daec2d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:41.445611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21fdc3-e983-443c-8b2c-1c7526c878ef None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:41.481210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:41.481210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:41.481210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:41.481210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:41.481695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=db842931b2394e8684012057347080af) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:41.481695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:41.481695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:41.495500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 267/537] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:41 2022] GET /identity/v3/role_assignments?user.id=db842931b2394e8684012057347080af&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:41.498824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:41.514880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:41.515805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:41.516188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:41.516570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:41.517472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:41.517882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:41.519429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:41.533104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 271/538] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:41.538934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:41.670377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc35f4a-bbd6-428f-9edc-9ad6b011336e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:41.708346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:41.709357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:41.709975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:41.710276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:41.717650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:41.719121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:41.742292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:41.743235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:41.744292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 268/539] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:41.750838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:41.892707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77998373-e954-4ce2-ac4c-72ba3447befb None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:41.934175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa14d7-1634-474f-996e-2f6fb5ce07f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:41.934883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa14d7-1634-474f-996e-2f6fb5ce07f6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:41.935167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa14d7-1634-474f-996e-2f6fb5ce07f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:41.935440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa14d7-1634-474f-996e-2f6fb5ce07f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:41.936242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa14d7-1634-474f-996e-2f6fb5ce07f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:41.936550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa14d7-1634-474f-996e-2f6fb5ce07f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:41.942266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa14d7-1634-474f-996e-2f6fb5ce07f6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:41.964733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 272/540] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:41.970862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:42.017545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:42.131420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b22ab2-d3d0-4918-9508-973c27feda9b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:42.180859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f898b7-efbd-47a0-a258-3298a130fe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:42.181580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f898b7-efbd-47a0-a258-3298a130fe70 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:42.181866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f898b7-efbd-47a0-a258-3298a130fe70 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:42.182156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f898b7-efbd-47a0-a258-3298a130fe70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:42.183025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f898b7-efbd-47a0-a258-3298a130fe70 None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=72fb6996e1974060b162070f8da365aa) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:42.183343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f898b7-efbd-47a0-a258-3298a130fe70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:42.184867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f898b7-efbd-47a0-a258-3298a130fe70 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:42.191349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 269/541] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:41 2022] GET /identity/v3/role_assignments?user.id=72fb6996e1974060b162070f8da365aa&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:42.587445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3db91-ddd7-4ebb-988d-2834be20f441 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:42.587988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3db91-ddd7-4ebb-988d-2834be20f441 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:42.587988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f3db91-ddd7-4ebb-988d-2834be20f441 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:42.589724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 273/542] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:42.595467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec296bf9-1cd8-4379-affc-2d2323ef8541 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:42.595872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec296bf9-1cd8-4379-affc-2d2323ef8541 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:42.596082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec296bf9-1cd8-4379-affc-2d2323ef8541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:42.618272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ec296bf9-1cd8-4379-affc-2d2323ef8541 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:42.619555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ec296bf9-1cd8-4379-affc-2d2323ef8541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:42.719196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 270/543] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:42.767169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3db91-ddd7-4ebb-988d-2834be20f441 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:42.827626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:42.906456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f3db91-ddd7-4ebb-988d-2834be20f441 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:42.948295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a312437-25ab-4871-ae0f-901ed481d449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:42.949104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a312437-25ab-4871-ae0f-901ed481d449 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:42.949413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a312437-25ab-4871-ae0f-901ed481d449 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:42.949691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a312437-25ab-4871-ae0f-901ed481d449 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:42.970061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a312437-25ab-4871-ae0f-901ed481d449 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5a312437-25ab-4871-ae0f-901ed481d449 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:42.978091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:42.978998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:42.980027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 274/544] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:42.985654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec296bf9-1cd8-4379-affc-2d2323ef8541 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:43.017615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:43.140226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec296bf9-1cd8-4379-affc-2d2323ef8541 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:43.185522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8912a13-c35b-42d2-abfc-d5bffa428280 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.185925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8912a13-c35b-42d2-abfc-d5bffa428280 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.186209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8912a13-c35b-42d2-abfc-d5bffa428280 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.187943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 275/545] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:43.188305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:43.188996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.189333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.189620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.190395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:43.190706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:43.192099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:43.192851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f3407-5a83-44d4-8b06-350f55072fdb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.193213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f3407-5a83-44d4-8b06-350f55072fdb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.193502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f3407-5a83-44d4-8b06-350f55072fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.209572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 271/546] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:43.210885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:43.211915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-163f3407-5a83-44d4-8b06-350f55072fdb None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:43.213102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-163f3407-5a83-44d4-8b06-350f55072fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:43.309655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:43.335223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 276/547] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:43.345327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dcfab5-5229-42b9-a55e-f11f88c4377a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:43.371217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f3407-5a83-44d4-8b06-350f55072fdb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:43.396698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:43.397740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.398024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.398305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.415436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:43.416903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:43.425803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:43.449958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:43.450869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:43.451718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:43.451718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:43.451718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:43.451718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:43.452513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 272/548] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:43.456544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:43.545000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f3407-5a83-44d4-8b06-350f55072fdb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:43.611602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f5d2f-8c02-4286-91ad-46e70f4faa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:43.612340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5d2f-8c02-4286-91ad-46e70f4faa06 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.612603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5d2f-8c02-4286-91ad-46e70f4faa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.612886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f5d2f-8c02-4286-91ad-46e70f4faa06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.631191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f5d2f-8c02-4286-91ad-46e70f4faa06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-8b9f5d2f-8c02-4286-91ad-46e70f4faa06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:43.639834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:43.640765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:43.642549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:43.647387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4899084c-4f19-4da6-b0d1-81fa21526af5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:43.650171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 277/549] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:43.654260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f5d2f-8c02-4286-91ad-46e70f4faa06 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:43.704647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:43.717804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:43.718535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.718829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.719106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.719869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:43.720179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:43.721682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:43.750165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 273/550] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:43.758265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:43.850547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:43.873957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f5d2f-8c02-4286-91ad-46e70f4faa06 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:43.905640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee7d29-2c67-418b-900b-dec0b458fce5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:43.948237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be793c2e-aaa4-4106-a6d1-0d9fc7516620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:43.948737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ee73c-83dc-4dfe-bdd8-339f65f1c12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:43.949485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ee73c-83dc-4dfe-bdd8-339f65f1c12f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.949778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ee73c-83dc-4dfe-bdd8-339f65f1c12f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.950050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ee73c-83dc-4dfe-bdd8-339f65f1c12f None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.950399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be793c2e-aaa4-4106-a6d1-0d9fc7516620 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.950824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ee73c-83dc-4dfe-bdd8-339f65f1c12f None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:43.951060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be793c2e-aaa4-4106-a6d1-0d9fc7516620 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.951178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ee73c-83dc-4dfe-bdd8-339f65f1c12f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:43.951657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be793c2e-aaa4-4106-a6d1-0d9fc7516620 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.952738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ee73c-83dc-4dfe-bdd8-339f65f1c12f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:43.961914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be793c2e-aaa4-4106-a6d1-0d9fc7516620 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:43.963810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be793c2e-aaa4-4106-a6d1-0d9fc7516620 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:43.965356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 278/551] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:43.967355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d38ea-e969-478c-bea9-c5d387be8ad7 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.969588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d38ea-e969-478c-bea9-c5d387be8ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.969881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d38ea-e969-478c-bea9-c5d387be8ad7 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:43.971496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 279/552] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:43.972238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d38ea-e969-478c-bea9-c5d387be8ad7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-be793c2e-aaa4-4106-a6d1-0d9fc7516620 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:43.985575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:43.986795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:43.988287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 274/553] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:09:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:43.989049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d7000-28b1-4dd1-97aa-c26d74a1035a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:43.989328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d7000-28b1-4dd1-97aa-c26d74a1035a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:43.989605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d7000-28b1-4dd1-97aa-c26d74a1035a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.017301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e97d7000-28b1-4dd1-97aa-c26d74a1035a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:44.018594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e97d7000-28b1-4dd1-97aa-c26d74a1035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:44.072900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:44.096262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 275/554] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:44.097117 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d7000-28b1-4dd1-97aa-c26d74a1035a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.114292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d38ea-e969-478c-bea9-c5d387be8ad7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.152547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8784c-8622-4fbc-92d9-8da4d77368d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.153507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8784c-8622-4fbc-92d9-8da4d77368d1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.153798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8784c-8622-4fbc-92d9-8da4d77368d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.154070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8784c-8622-4fbc-92d9-8da4d77368d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.165204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8784c-8622-4fbc-92d9-8da4d77368d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:44.166678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8784c-8622-4fbc-92d9-8da4d77368d1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-2ab8784c-8622-4fbc-92d9-8da4d77368d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:44.183981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:44.185045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:44.186016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:44.186016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:44.186016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:44.186016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:44.186016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 280/555] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:44.186956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c51b-4c04-4e0f-b7f9-60f37720cbd2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.187251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c51b-4c04-4e0f-b7f9-60f37720cbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.187524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c51b-4c04-4e0f-b7f9-60f37720cbd2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.189130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 281/556] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:44.190026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c51b-4c04-4e0f-b7f9-60f37720cbd2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.202647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:44.217993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d7000-28b1-4dd1-97aa-c26d74a1035a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.256435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.257568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.258094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.258435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.259405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:44.259992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:44.261478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:44.280501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 276/557] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:44.281447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.309100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c51b-4c04-4e0f-b7f9-60f37720cbd2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.329784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:44.347901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc3a19-776a-4e17-8afd-82d624731c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.348708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc3a19-776a-4e17-8afd-82d624731c17 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.348984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc3a19-776a-4e17-8afd-82d624731c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.349478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc3a19-776a-4e17-8afd-82d624731c17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.359955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc3a19-776a-4e17-8afd-82d624731c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-34fc3a19-776a-4e17-8afd-82d624731c17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:44.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:44.368864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 282/558] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:44.370208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43deccc-ae1b-4585-851f-fc62147be21a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.370982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43deccc-ae1b-4585-851f-fc62147be21a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.370982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43deccc-ae1b-4585-851f-fc62147be21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.387942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e43deccc-ae1b-4585-851f-fc62147be21a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:44.389203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e43deccc-ae1b-4585-851f-fc62147be21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:44.406177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4436f3b-c698-47b9-aecb-09869199377e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.442563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.443295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.443626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.443909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.444627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:44.444935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:44.446348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:44.453824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:44.464902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 277/559] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:44.464902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.467518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 283/560] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:44.468531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43deccc-ae1b-4585-851f-fc62147be21a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.586973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43deccc-ae1b-4585-851f-fc62147be21a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.594568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:44.599612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f76578-a757-470f-bd4d-146985592ea4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.627427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.628195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.628508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.628757 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.629562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:44.629891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:44.631731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:44.636525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16bcaf-b67f-494b-bbbb-6be50afe9136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.637211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16bcaf-b67f-494b-bbbb-6be50afe9136 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.637489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16bcaf-b67f-494b-bbbb-6be50afe9136 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.637766 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16bcaf-b67f-494b-bbbb-6be50afe9136 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.641542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 284/561] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:44.642657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.705291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:44.716680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16bcaf-b67f-494b-bbbb-6be50afe9136 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:44.718164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16bcaf-b67f-494b-bbbb-6be50afe9136 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:44.750234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caffec-7f93-41d2-84ea-d6decb5ae2ae None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.793023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.794049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.794331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.794598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.805778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:44.807752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:44.823272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:44.824330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:44.825394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:09:44.826374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:44.829320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 285/562] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:44.830470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 278/563] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:44.830670 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.831120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16bcaf-b67f-494b-bbbb-6be50afe9136 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.932217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16bcaf-b67f-494b-bbbb-6be50afe9136 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.934724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ea454-a3db-4a15-acf6-b1ee7f9d8228 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:44.951836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:44.968192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.968963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.969270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.969545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.969991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b9eba-315b-4616-aa40-490d0c04a89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:44.970851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b9eba-315b-4616-aa40-490d0c04a89c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.971132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b9eba-315b-4616-aa40-490d0c04a89c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.971407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b9eba-315b-4616-aa40-490d0c04a89c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.980202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:44.981814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:44.982715 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b9eba-315b-4616-aa40-490d0c04a89c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-436b9eba-315b-4616-aa40-490d0c04a89c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:44.991100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:44.992184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:44.993093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 286/564] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:44.993949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b176ebf-27e1-42df-94ab-530916afd8f9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:44.994225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b176ebf-27e1-42df-94ab-530916afd8f9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:44.994510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b176ebf-27e1-42df-94ab-530916afd8f9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:44.997291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 287/565] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:44.998195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b176ebf-27e1-42df-94ab-530916afd8f9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:44.999991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:45.001044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:45.002192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:45.002192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:45.002192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:45.002192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:45.002192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 279/566] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:45.002510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.054768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.104909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bb919-d47b-45cc-83c7-5ba38e737ba0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.110804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b176ebf-27e1-42df-94ab-530916afd8f9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.140143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc265a-cf5f-4f84-8e60-2bc149c34fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.140950 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc265a-cf5f-4f84-8e60-2bc149c34fde None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.141267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc265a-cf5f-4f84-8e60-2bc149c34fde None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.141549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc265a-cf5f-4f84-8e60-2bc149c34fde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.145126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.145846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.146288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.146576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.147473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:45.147781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.149343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.152949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc265a-cf5f-4f84-8e60-2bc149c34fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.154501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc265a-cf5f-4f84-8e60-2bc149c34fde None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.167895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 288/567] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:09:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:45.168697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-54bc265a-cf5f-4f84-8e60-2bc149c34fde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:45.172554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:45.173641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:45.174757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:45.174757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:45.174757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:45.174757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:45.174757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 280/568] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:45.175108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ecde7-abc1-44b1-95f9-6c4a0390bc1d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.175198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ecde7-abc1-44b1-95f9-6c4a0390bc1d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.175506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ecde7-abc1-44b1-95f9-6c4a0390bc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.191565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-995ecde7-abc1-44b1-95f9-6c4a0390bc1d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:45.192887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-995ecde7-abc1-44b1-95f9-6c4a0390bc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:45.208436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.261929 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 281/569] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:45.262466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ecde7-abc1-44b1-95f9-6c4a0390bc1d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.277933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f24e67-e472-412b-87e1-8b6629e4f300 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.293753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.311799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.312544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.312830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.313107 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.313912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:45.314229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.315634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.324101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 289/570] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:45.324813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.368121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ecde7-abc1-44b1-95f9-6c4a0390bc1d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.402365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.403028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.403320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.403596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.404324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:45.404628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.406089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.418181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 282/571] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:45.419035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.427841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.432322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08b5b5-a759-4464-bbd2-bc2b56ebfa7d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.467230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.468049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.468348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.468611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.482117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.483695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:45.503150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:45.504250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:45.505247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:45.505247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:45.505247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:45.505247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:45.505247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 290/572] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:45.505760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.514109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.524488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe892ca-ebec-401a-8c0e-f3b6dd17a48d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.558429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.559103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.559393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.559682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.560409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:45.560719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.562185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.575317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 283/573] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:45.576273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.607066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717daa1-4a3a-40a1-b790-1392f73c0e51 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.639420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c625bf-b6be-446d-9e5c-56f93ab6c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.640180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c625bf-b6be-446d-9e5c-56f93ab6c603 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.640459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c625bf-b6be-446d-9e5c-56f93ab6c603 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.640731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c625bf-b6be-446d-9e5c-56f93ab6c603 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.650861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c625bf-b6be-446d-9e5c-56f93ab6c603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-88c625bf-b6be-446d-9e5c-56f93ab6c603 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:45.658491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:45.659536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:45.660384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 291/574] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:45.661087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c625bf-b6be-446d-9e5c-56f93ab6c603 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.673608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.680754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4415ed6-b348-4d73-b37f-793b8bf2fc30 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.718091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.719307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.719643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.719922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.730644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.732145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:45.750051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:45.751152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:45.752149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:45.752149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:45.752149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:45.752149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:45.752149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 284/575] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:45.752449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.755842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.772786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c625bf-b6be-446d-9e5c-56f93ab6c603 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.809423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.810241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.810528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.810807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.818484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.820014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:45.834021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:45.835389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 292/576] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:45.836636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.858097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377f252-0ebe-4b2f-b692-ea7ca5ada744 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.894523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.895280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.895565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.895835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.896643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:45.896997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.898424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:45.910035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 285/577] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:45.910865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:45.912164 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.944267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d459ce45-5943-4df6-a515-8df159ebb574 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:45.973885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:45.979710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:45.980558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:45.980837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:45.981114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:45.987947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:45.989431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:46.003916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:46.004873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 293/578] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:46.006044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.016626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9689e-3bf3-4a0b-82c2-ea3e39a87875 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.053036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.053733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.054018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.054291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.055116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:46.055419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.057054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.065158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 286/579] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:46.066026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.114836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956112a-554f-4fc4-a2ca-94abd9b7ac6f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.142292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:46.148775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.149472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.149765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.150025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.150749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:46.151050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.152444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.164446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 294/580] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:46.165298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.171778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1687f1-a158-4938-af3f-f9a116d565fe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.198796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:46.210799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.211512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.211795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.212099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.212946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:46.213291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.214763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.232386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 287/581] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:46.233203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.267353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2af190-a846-44f4-af0e-77253fb7b3b6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.301381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.302177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.302527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.302861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.309760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.311184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:46.325200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:46.326206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:46.327142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 295/582] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:46.327626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.340249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ec90b-e2b6-46a5-bb78-48c35b6bb554 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.375130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.375817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.376081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.376355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.377140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:46.377484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.378914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:46.379034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.390581 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 288/583] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:46.391371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.432081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c3eee-abc4-4a2f-a470-efffafd4168b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.437133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:46.466449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.467301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.467582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.467852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.478396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.479876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.492900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4029b759-4ec7-41dc-a963-aa9e5549addc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:46.498097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:46.499069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:46.500097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:46.500097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:46.500097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:46.500097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:46.500097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 296/584] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:46.500708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.537222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.537836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.538109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.538378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.546125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.547597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:46.561524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:46.562466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 289/585] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:46.563437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.604227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264c604-fa36-4b2b-ba46-1c669ab7db50 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.605931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:46.640308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.641139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.641451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.641723 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.652026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.653510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.666650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494fae6-c767-4ef5-905c-0192a1312e89 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.668912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:46.671339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:46.672276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:46.673210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:46.673210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:46.673210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:46.673210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:46.673210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 297/586] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:46.673528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.704508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.705253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.705536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.705811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.706593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:46.706899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.708336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.720491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 290/587] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:46.721312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.781667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65995eb-2f8e-403f-9d13-0a128d3a0973 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.819767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.820372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.820654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.820931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.821803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=0f15957f640a411eb4ac8861454f2b97) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:46.822117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.822381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4deda-da08-43b1-bf16-68ea861f67f6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.823550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.829066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 298/588] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/role_assignments?user.id=0f15957f640a411eb4ac8861454f2b97&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:46.829884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.854063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:46.869251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.869942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.870225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.870791 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:46.871239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:46.871545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:46.872987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:46.885857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 291/589] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:46.886645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:46.916782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:46.955800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d5c84-30b0-4311-8cbc-7ad68a90f7a4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:46.997876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:46.998902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:46.999353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:46.999634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.000503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:47.000825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.002391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.012441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551c1229-0cb9-44fc-84d5-266e7551f3a4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.022584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 299/590] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:47.023415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.053778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.054448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.054733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.055010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.056178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:47.056491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.057883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.074468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 292/591] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:47.074468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.127340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:47.154846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242abc0e-894b-43fb-9be7-92c4664e0394 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.166364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:47.184597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab37a5-c9ab-4653-84ab-2283a7b13ef7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.190483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30db49-7fa5-4002-93f6-a1b90f3c5cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.191400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30db49-7fa5-4002-93f6-a1b90f3c5cd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.191720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30db49-7fa5-4002-93f6-a1b90f3c5cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.191953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30db49-7fa5-4002-93f6-a1b90f3c5cd8 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/e3241452a88a40da9c9b8fa79c4fc9c0/roles/3b84c0487e23407e8452c17b5886b640` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.207816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30db49-7fa5-4002-93f6-a1b90f3c5cd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=e3241452a88a40da9c9b8fa79c4fc9c0, role_id=3b84c0487e23407e8452c17b5886b640)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.209938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30db49-7fa5-4002-93f6-a1b90f3c5cd8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.223669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.233468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.233699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 300/592] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:47 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/e3241452a88a40da9c9b8fa79c4fc9c0/roles/3b84c0487e23407e8452c17b5886b640 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:47.233805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.233805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.233972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30db49-7fa5-4002-93f6-a1b90f3c5cd8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.243092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.244748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:47.261551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:47.262645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:47.263786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 293/593] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:47.264821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.367270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30db49-7fa5-4002-93f6-a1b90f3c5cd8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.400695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:47.404834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b481b8-d5c4-4a74-b2a0-fef77103b8f3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.415982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.417657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.417803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.417978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.429908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.431405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:47.452241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:47.453301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:47.454211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:47.454211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:47.454211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:47.454211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:47.454456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.454456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.454456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.454456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.458378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 301/594] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:47.459397 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.465141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.472499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:47.488153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:47.489129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 294/595] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:47.490916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.494304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:47.588083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47813342-2c85-4a7b-804c-6bd921c24487 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.619128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47f3bb-ca0f-4b17-8a9e-00fa7b518c44 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.625851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.626781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.627220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.627661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.628687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:47.629166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.630881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.657228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 302/596] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Jun 16 17:09:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:47.657228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.665756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.666690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.667181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.667600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.668568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:47.669039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.670649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.689449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 295/597] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:47.690652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.694475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:47.761005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:47.784097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ddb0b0-14c1-46c3-a3d7-54ce1a276d72 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.813736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761722e-a8c2-4568-8a65-7c84e527b29f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:47.831331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.832015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.832300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.832578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.833798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:47.834120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.836278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.853033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 303/598] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:47.853935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.854812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:47.855858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:47.856428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:47.856962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:47.870813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:47.872580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:47.891518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:47.892500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:47.893467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:47.893467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:47.893467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:47.893467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:47.900139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 296/599] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:47.901363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:47.973047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:47.980851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39402bce-51ad-4f63-9a53-adc0dd23b0bc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:48.020472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:48.022635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:48.022926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:48.023645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:48.026258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9558287d-67d8-4c25-b34f-c539c1fbee7b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:48.034351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:48.035871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:48.046618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:48.061695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:48.062868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:48.064023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:48.064023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:48.064023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:48.064023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:48.064023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:48.064023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:48.064023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:48.064023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:48.067083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 304/600] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:48.068845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:48.070128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16a0e3-46d2-4c4c-9bce-2d691b03deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:48.071166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16a0e3-46d2-4c4c-9bce-2d691b03deb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:48.071521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16a0e3-46d2-4c4c-9bce-2d691b03deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:48.071895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16a0e3-46d2-4c4c-9bce-2d691b03deb0 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/d6179dc2bd1d4ff2a04a8362d63e7b7c/roles/3b84c0487e23407e8452c17b5886b640` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:48.090374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16a0e3-46d2-4c4c-9bce-2d691b03deb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=d6179dc2bd1d4ff2a04a8362d63e7b7c, role_id=3b84c0487e23407e8452c17b5886b640)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:48.092100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c16a0e3-46d2-4c4c-9bce-2d691b03deb0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:48.113218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 297/601] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:47 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/d6179dc2bd1d4ff2a04a8362d63e7b7c/roles/3b84c0487e23407e8452c17b5886b640 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:48.113218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16a0e3-46d2-4c4c-9bce-2d691b03deb0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:48.223827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c16a0e3-46d2-4c4c-9bce-2d691b03deb0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:48.257818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f23d5d-eb9d-4676-8c71-e87d9ff5b69b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:48.261410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:48.261886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:48.262172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:48.262447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:48.263238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:48.263546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:48.264972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:48.278401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 298/602] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:48.284281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:48.339689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:48.340427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:48.340709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:48.340982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:48.341800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:48.342110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:48.343582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:48.350280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:48.369116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 305/603] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:48.374533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:48.407959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:48.476394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2167-df49-44fc-aa57-84c4a89c0bc4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:48.502452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df2f16-8367-4f84-ac05-4a2bf9d06a21 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:48.536412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ebbf-8f92-40dd-bf35-374aa741e2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:48.537924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ebbf-8f92-40dd-bf35-374aa741e2c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:48.538403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ebbf-8f92-40dd-bf35-374aa741e2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:48.538973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498ebbf-8f92-40dd-bf35-374aa741e2c2 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/db842931b2394e8684012057347080af/roles/3b84c0487e23407e8452c17b5886b640` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:48.544453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:48.545359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:48.545649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:48.545926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:48.560902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:48.562838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:48.564825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498ebbf-8f92-40dd-bf35-374aa741e2c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=db842931b2394e8684012057347080af, role_id=3b84c0487e23407e8452c17b5886b640)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:48.566566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498ebbf-8f92-40dd-bf35-374aa741e2c2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:48.591910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:48.592898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:48.593882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:48.593882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:48.593882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:48.593882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:48.593882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:48.593882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:48.593882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:48.593882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:48.599182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 299/604] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:48 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/db842931b2394e8684012057347080af/roles/3b84c0487e23407e8452c17b5886b640 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:48.600947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 306/605] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:48.609833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ebbf-8f92-40dd-bf35-374aa741e2c2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:48.730487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:48.763521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498ebbf-8f92-40dd-bf35-374aa741e2c2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:48.811493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c04b67-3d01-4017-bacb-64b9f7302797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:48.812194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c04b67-3d01-4017-bacb-64b9f7302797 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:48.812475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c04b67-3d01-4017-bacb-64b9f7302797 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:48.812748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c04b67-3d01-4017-bacb-64b9f7302797 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:48.813602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c04b67-3d01-4017-bacb-64b9f7302797 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:48.813909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c04b67-3d01-4017-bacb-64b9f7302797 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:48.815370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c04b67-3d01-4017-bacb-64b9f7302797 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:48.831280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 300/606] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:48.837477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:48.934580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:48.964265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a3624-99af-4fe5-aec5-1a14868776b2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:49.007081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab1e95e-4a00-407e-b984-8fb65c4a14cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:49.007913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1e95e-4a00-407e-b984-8fb65c4a14cd None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:49.008198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1e95e-4a00-407e-b984-8fb65c4a14cd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:49.008477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab1e95e-4a00-407e-b984-8fb65c4a14cd None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/72fb6996e1974060b162070f8da365aa/roles/3b84c0487e23407e8452c17b5886b640` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:49.030159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1e95e-4a00-407e-b984-8fb65c4a14cd None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=72fb6996e1974060b162070f8da365aa, role_id=3b84c0487e23407e8452c17b5886b640)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:49.032504 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab1e95e-4a00-407e-b984-8fb65c4a14cd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:49.053724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 307/607] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:48 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/72fb6996e1974060b162070f8da365aa/roles/3b84c0487e23407e8452c17b5886b640 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:49.154184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff4b3-417a-43e1-86d7-9ea2f394a1a5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:49.154519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff4b3-417a-43e1-86d7-9ea2f394a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:49.154748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afff4b3-417a-43e1-86d7-9ea2f394a1a5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:49.156130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 301/608] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:49.162140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff4583-0f5a-474b-8b1f-713bf87b245d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:49.162473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff4583-0f5a-474b-8b1f-713bf87b245d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:49.162849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff4583-0f5a-474b-8b1f-713bf87b245d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:49.186528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-56ff4583-0f5a-474b-8b1f-713bf87b245d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:49.187932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-56ff4583-0f5a-474b-8b1f-713bf87b245d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:49.271750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 308/609] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:49.306472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afff4b3-417a-43e1-86d7-9ea2f394a1a5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:49.451455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afff4b3-417a-43e1-86d7-9ea2f394a1a5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:49.497064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c063f0-d6e7-4a67-805a-21585876fc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:49.497942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c063f0-d6e7-4a67-805a-21585876fc62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:49.498225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c063f0-d6e7-4a67-805a-21585876fc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:49.498501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c063f0-d6e7-4a67-805a-21585876fc62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:49.501521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:49.512855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c063f0-d6e7-4a67-805a-21585876fc62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-57c063f0-d6e7-4a67-805a-21585876fc62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:49.520620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:49.522105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:49.523010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 302/610] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:49.530158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff4583-0f5a-474b-8b1f-713bf87b245d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:49.623107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baf07b-873d-4874-ab32-b0baef95a2be None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:49.623462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baf07b-873d-4874-ab32-b0baef95a2be None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:49.623674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53baf07b-873d-4874-ab32-b0baef95a2be None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:49.625054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 303/611] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:49.636214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934621-01ac-4119-b510-ab7684b4ec9f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:49.636442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:49.636542 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934621-01ac-4119-b510-ab7684b4ec9f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:49.636758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18934621-01ac-4119-b510-ab7684b4ec9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:49.674647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-18934621-01ac-4119-b510-ab7684b4ec9f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:49.677662 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-18934621-01ac-4119-b510-ab7684b4ec9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:49.694502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff4583-0f5a-474b-8b1f-713bf87b245d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:49.761738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:49.762668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:49.763180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:49.763600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:49.764559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:49.766220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:49.767765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:49.785857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 309/612] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:49.788409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:49.822696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 304/613] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:09:49.865261 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934621-01ac-4119-b510-ab7684b4ec9f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:49.976759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dc6f2-3f9a-4a11-80a3-b500f5895aef None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:50.017424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:50.038523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18934621-01ac-4119-b510-ab7684b4ec9f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:50.051477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:50.052756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:50.053043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:50.053375 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:50.074120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:50.076058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cba40-6789-4f91-8471-17e5eb1c44aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:50.076539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:50.077475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cba40-6789-4f91-8471-17e5eb1c44aa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:50.077983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cba40-6789-4f91-8471-17e5eb1c44aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:50.078493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9cba40-6789-4f91-8471-17e5eb1c44aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:50.078736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:50.090751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9cba40-6789-4f91-8471-17e5eb1c44aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0f9cba40-6789-4f91-8471-17e5eb1c44aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:50.099090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:50.100202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:50.101088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:50.104042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 305/614] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:50.109981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:50.110880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:50.111809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:50.111809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:50.111809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:50.111809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:50.111809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 310/615] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:50.116415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cba40-6789-4f91-8471-17e5eb1c44aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:50.117731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:50.252102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3947b1-3322-44fc-903f-3f2adcba11ad None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:50.287403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:50.288064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:50.288341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:50.288663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:50.289364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:50.289672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:50.291046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:50.304481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 311/616] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:50.309636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:50.341551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:50.411445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019ab9d-729a-460c-a9c1-ec096b42857a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:50.447010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:50.447897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:50.448242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:50.448579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:50.455119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9cba40-6789-4f91-8471-17e5eb1c44aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:50.460958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:50.463413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:50.493949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:50.495062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:50.496025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 312/617] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:09:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:50.502848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:50.519512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:50.520205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:50.520489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:50.520768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:50.521602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:50.521912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:50.523316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:50.532773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 306/618] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:50.540021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:50.617769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:50.629742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ab828-e68d-4a40-992c-9f0b67eb59a9 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:50.652965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:50.697608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:50.698377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:50.698658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:50.698929 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:50.699773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:50.700080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:50.701545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:50.733620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3282ee-30c2-4607-807a-0696ac16ec6c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:50.747525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 313/619] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Jun 16 17:09:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:50.753330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:50.798641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:50.799366 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:50.799644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:50.799915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:50.825827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:50.827346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:50.851941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:50.852983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:50.854364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:50.854364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:50.854364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:50.854364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:50.854364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 307/620] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:50.859795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:50.950292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba9dc4a-526b-4cf9-a467-12e6febe4f71 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:50.986702 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa7de3a-efb3-4805-b0eb-6ba84d406fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:50.987507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7de3a-efb3-4805-b0eb-6ba84d406fec None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:50.987776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7de3a-efb3-4805-b0eb-6ba84d406fec None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:50.988045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa7de3a-efb3-4805-b0eb-6ba84d406fec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:50.996890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:50.999873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ee0266-3ba5-4d6b-9df1-b523be42ff8d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.002008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7de3a-efb3-4805-b0eb-6ba84d406fec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.003560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa7de3a-efb3-4805-b0eb-6ba84d406fec None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.012935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-aaa7de3a-efb3-4805-b0eb-6ba84d406fec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:51.024672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:51.025771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:51.027449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:51.027449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:51.027449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:51.027449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:51.027449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 314/621] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:51.032728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e175d25-cad9-45cf-b026-1cede6e1490d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.033117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e175d25-cad9-45cf-b026-1cede6e1490d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.033429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e175d25-cad9-45cf-b026-1cede6e1490d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.035092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 315/622] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:51.035807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e175d25-cad9-45cf-b026-1cede6e1490d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.043682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729baf4-b44b-43da-bdf7-93a9e32e83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.044403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729baf4-b44b-43da-bdf7-93a9e32e83b0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.044690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729baf4-b44b-43da-bdf7-93a9e32e83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.044978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729baf4-b44b-43da-bdf7-93a9e32e83b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.045763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729baf4-b44b-43da-bdf7-93a9e32e83b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:51.046086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729baf4-b44b-43da-bdf7-93a9e32e83b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.047534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729baf4-b44b-43da-bdf7-93a9e32e83b0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.065977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 308/623] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:51.067274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092a7f-9b56-49f7-98d3-dabad7ebb33a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.067571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092a7f-9b56-49f7-98d3-dabad7ebb33a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.067887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c092a7f-9b56-49f7-98d3-dabad7ebb33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.087097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3c092a7f-9b56-49f7-98d3-dabad7ebb33a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:51.088321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3c092a7f-9b56-49f7-98d3-dabad7ebb33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:51.149979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e175d25-cad9-45cf-b026-1cede6e1490d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.173654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 309/624] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:51.174627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c092a7f-9b56-49f7-98d3-dabad7ebb33a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.188017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.188684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.189009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.189318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.190094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:51.190488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.192015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.212222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 316/625] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:51.213653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.284457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:51.302769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:51.313452 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c092a7f-9b56-49f7-98d3-dabad7ebb33a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.332291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c096a87-b156-419a-b821-0c9d38a898cd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.355517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.356476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.356762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.357112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.365775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.366652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea0ba1-23f2-41e7-84fb-2dcf9b34f22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.367244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.367460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea0ba1-23f2-41e7-84fb-2dcf9b34f22c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.367739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea0ba1-23f2-41e7-84fb-2dcf9b34f22c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.368011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea0ba1-23f2-41e7-84fb-2dcf9b34f22c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.374818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea0ba1-23f2-41e7-84fb-2dcf9b34f22c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.376245 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea0ba1-23f2-41e7-84fb-2dcf9b34f22c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:51.383191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:51.385466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:09:51.387800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:51.389290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 310/626] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:51.390008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3fea0ba1-23f2-41e7-84fb-2dcf9b34f22c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:51.391196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:51.393563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:51.397223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 317/627] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:51.398559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675379b0-1524-4eda-8e04-2c009316be39 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.398804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675379b0-1524-4eda-8e04-2c009316be39 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.399086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675379b0-1524-4eda-8e04-2c009316be39 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.405002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 318/628] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:51.405002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675379b0-1524-4eda-8e04-2c009316be39 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.514443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675379b0-1524-4eda-8e04-2c009316be39 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.515530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1d0216-b7f2-44bb-bb24-681ccc840e34 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.538149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:51.555664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.556491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.556821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.557102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.557996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:51.558293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.559709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467d555-080c-40b1-a153-b1bdf13b4024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.559869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.560541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467d555-080c-40b1-a153-b1bdf13b4024 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.560819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467d555-080c-40b1-a153-b1bdf13b4024 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.561092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467d555-080c-40b1-a153-b1bdf13b4024 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.575452 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467d555-080c-40b1-a153-b1bdf13b4024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:51.587417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 319/629] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:51.587417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e467d555-080c-40b1-a153-b1bdf13b4024 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:51.587681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:51.588619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 311/630] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:51.589531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518125-57bd-424e-9b55-d5534d0d3b01 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.590331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518125-57bd-424e-9b55-d5534d0d3b01 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.590331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83518125-57bd-424e-9b55-d5534d0d3b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.606966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-83518125-57bd-424e-9b55-d5534d0d3b01 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:51.608144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-83518125-57bd-424e-9b55-d5534d0d3b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:51.612276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:51.679652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 312/631] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:51.680821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83518125-57bd-424e-9b55-d5534d0d3b01 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.699571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e1cf3-d56a-42f9-9b51-1056bdce2eb5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.735058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.735745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.736015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.736301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.737091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:51.737424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.738855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.750535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 320/632] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:51.751315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.783672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83518125-57bd-424e-9b55-d5534d0d3b01 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.792089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:51.819410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e570ab-ec06-4631-96f1-f1166d69a973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.820091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e570ab-ec06-4631-96f1-f1166d69a973 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.820374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e570ab-ec06-4631-96f1-f1166d69a973 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.820648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e570ab-ec06-4631-96f1-f1166d69a973 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.821439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e570ab-ec06-4631-96f1-f1166d69a973 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:51.821745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e570ab-ec06-4631-96f1-f1166d69a973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.823138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e570ab-ec06-4631-96f1-f1166d69a973 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.831815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 313/633] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:51.833077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e308-6c04-4a3e-84fe-2bd018ba8ed2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.834324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e308-6c04-4a3e-84fe-2bd018ba8ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.834324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e308-6c04-4a3e-84fe-2bd018ba8ed2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.837209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 314/634] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:51.837209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e308-6c04-4a3e-84fe-2bd018ba8ed2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:51.844134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:51.861549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1263113-f322-4b73-a657-e6cc81bf8188 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.897031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb55fb-692e-4675-a654-1c3df8a9fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.897714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb55fb-692e-4675-a654-1c3df8a9fca5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.897994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb55fb-692e-4675-a654-1c3df8a9fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.898266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb55fb-692e-4675-a654-1c3df8a9fca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.947838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e308-6c04-4a3e-84fe-2bd018ba8ed2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:51.967044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb55fb-692e-4675-a654-1c3df8a9fca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:51.968534 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb55fb-692e-4675-a654-1c3df8a9fca5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:51.983799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911fb8d-17fa-4477-93b0-b7c2d26fee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:51.984625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911fb8d-17fa-4477-93b0-b7c2d26fee9c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:51.984912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911fb8d-17fa-4477-93b0-b7c2d26fee9c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:51.985193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911fb8d-17fa-4477-93b0-b7c2d26fee9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:51.995969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911fb8d-17fa-4477-93b0-b7c2d26fee9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7911fb8d-17fa-4477-93b0-b7c2d26fee9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:52.003821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:52.004776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 315/635] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:52.006271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911fb8d-17fa-4477-93b0-b7c2d26fee9c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.007882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:52.069415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:52.073777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 321/636] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:52.074535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb55fb-692e-4675-a654-1c3df8a9fca5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.113683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911fb8d-17fa-4477-93b0-b7c2d26fee9c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.152079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9180a-b281-429b-9063-e6146a2e8fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.152999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9180a-b281-429b-9063-e6146a2e8fd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.153315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9180a-b281-429b-9063-e6146a2e8fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.153598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9180a-b281-429b-9063-e6146a2e8fd5 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/0f15957f640a411eb4ac8861454f2b97/roles/3b84c0487e23407e8452c17b5886b640` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.168852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9180a-b281-429b-9063-e6146a2e8fd5 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=0f15957f640a411eb4ac8861454f2b97, role_id=3b84c0487e23407e8452c17b5886b640)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:52.170350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9180a-b281-429b-9063-e6146a2e8fd5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:52.179035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb55fb-692e-4675-a654-1c3df8a9fca5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.184735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 316/637] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:09:52 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/0f15957f640a411eb4ac8861454f2b97/roles/3b84c0487e23407e8452c17b5886b640 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:09:52.185687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a062c2-ce85-4296-a1ad-a390753bb740 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.185978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a062c2-ce85-4296-a1ad-a390753bb740 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.186258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a062c2-ce85-4296-a1ad-a390753bb740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.205485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c3a062c2-ce85-4296-a1ad-a390753bb740 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:52.206817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c3a062c2-ce85-4296-a1ad-a390753bb740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:52.228483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.229632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.229912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.230193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.238917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:52.245408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:52.246954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:52.266850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:52.268033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:52.268960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:52.268960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:52.268960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:52.268960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:52.268960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 322/638] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:52.268960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.305205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 317/639] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:52.305205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a062c2-ce85-4296-a1ad-a390753bb740 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.360654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:52.387509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601f904-e1e5-47a2-a9f1-a99fa3668dc5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.425549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.426531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.426968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.427328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.428433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:52.428745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:52.430165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:52.432766 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a062c2-ce85-4296-a1ad-a390753bb740 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.438450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 323/640] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:52.439096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.479288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.480481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.480882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.481054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.494361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:52.495872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:52.504048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:52.515701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:52.516824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:52.518020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:52.518020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:52.518020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:52.518020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:52.524481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 318/641] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:52.524481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.556821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f886a6e-273a-4e1b-bf6e-99b224f836d2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.594747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.595578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.595970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.596307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.597150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:52.597517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:52.599019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:52.611246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 324/642] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:52.612123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.632416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:52.639799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19c4cb-f11f-4e2d-97f2-6251b8c487bd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.676603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8501e4-3e83-42b8-ab75-8b42fb4f6a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.678139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8501e4-3e83-42b8-ab75-8b42fb4f6a0c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.678441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8501e4-3e83-42b8-ab75-8b42fb4f6a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.678722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8501e4-3e83-42b8-ab75-8b42fb4f6a0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.689367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8501e4-3e83-42b8-ab75-8b42fb4f6a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cc8501e4-3e83-42b8-ab75-8b42fb4f6a0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:52.697968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:52.704411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 319/643] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:52.705639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8501e4-3e83-42b8-ab75-8b42fb4f6a0c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.730590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9ef9-6c01-43af-bc86-fe33bed0e873 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.745553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:52.768737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.769693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.770045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.770452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.782000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:52.783469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:52.801793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:52.802821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:52.804094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:52.804094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:52.804094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:52.804094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:52.807723 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 325/644] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:52.808636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.814340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8501e4-3e83-42b8-ab75-8b42fb4f6a0c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.852862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.853634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.853867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.854141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.854898 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:52.855200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:52.857065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:52.870640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 320/645] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:52.871453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:52.889584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:52.925280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd362c-548d-44cd-8230-790715b0b566 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.960580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:52.961380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:52.961680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:52.961928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:52.969380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:52.970816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:52.984883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267722eb-1060-4358-a84b-7ee263a55ddb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:52.985832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:52.986775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 326/646] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:52.987773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.014566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:53.033272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.034040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.034320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.034598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.035350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:53.035701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.037166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.050529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 321/647] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:53.050529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.099201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cce62-6e88-427e-b60b-dd299e32e4fc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.128859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:53.134319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.135147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.135597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.136028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.136888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:53.137357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.138850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.150517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 327/648] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:53.151538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.159048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e53c9e-6f8b-4d96-809a-23fac615c482 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.193167 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.194007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.194269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.194549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.202163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.203629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:53.218533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:53.219468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:53.220385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 322/649] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:53.221518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.248555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:53.267184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e42690-6680-4156-9cd7-4fce9f847329 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.309476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.310147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.310425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.310699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.311534 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:53.311843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.313282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.325664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc78346-ad4f-4189-b34d-a7ca41d2cc72 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.332421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 328/650] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:53.333150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.367818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.369123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.369434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.369711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.372423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:53.382428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.383872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:53.401885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:53.402829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:53.403750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:53.403750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:53.403750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:53.403750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:53.403750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 323/651] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:53.404044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.444089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e20f6-c77a-4c1f-9b30-07f108a0e1ce None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.485094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.486746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.487096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.487441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.496027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.497613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:53.513083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:53.514066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 17:09:53.515103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:53.516186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ecf04-63fb-4f2a-8c8f-dfcf296b1284 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.518942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 329/652] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:53.519705 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.523201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:53.556576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.557284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.557820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.558110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.559769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:53.560090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.561554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.581737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 324/653] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:53.581737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.631509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb26f37-5025-4054-b10a-8d457e702654 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.635858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:53.667814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f95f1-8d2d-4cdc-9c20-f4092f1bb778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.668456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f95f1-8d2d-4cdc-9c20-f4092f1bb778 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.668737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f95f1-8d2d-4cdc-9c20-f4092f1bb778 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.669011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f95f1-8d2d-4cdc-9c20-f4092f1bb778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.690732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6704e65b-0a1e-4c40-ab52-580ed15ec9a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.727974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.728666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.729030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.729409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.730216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:53.730596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.732065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.742359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f95f1-8d2d-4cdc-9c20-f4092f1bb778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.743882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f95f1-8d2d-4cdc-9c20-f4092f1bb778 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.749735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 325/654] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:53.749885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:53.749976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.858114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 330/655] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:53.862695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f95f1-8d2d-4cdc-9c20-f4092f1bb778 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.865505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca2972-1df0-44b3-a9e2-a629acb7e071 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.875367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:53.904544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:53.905250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:53.905533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:53.905811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:53.906646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:53.906953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:53.908393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:53.930549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 326/656] 10.222.0.66 () {56 vars in 1137 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:53.930549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:53.977881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f95f1-8d2d-4cdc-9c20-f4092f1bb778 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:53.979461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:54.015941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.016706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.016992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.017309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.018201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=e3241452a88a40da9c9b8fa79c4fc9c0) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:54.018445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:54.020060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:54.037332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 331/657] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/role_assignments?user.id=e3241452a88a40da9c9b8fa79c4fc9c0&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:54.037332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:54.043168 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96aec7-756e-4e33-aa47-ec693c162a68 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.080667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.081621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.081972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.082314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.090935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:54.092421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:54.108232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:54.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 327/658] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:54.110345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:54.148577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:54.159608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0af13-94e3-4cd7-95d0-90abd404244c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.198333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.199311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.199727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.200139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.212756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:54.214471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:54.234179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:54.235206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:54.236186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:54.236186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:54.236186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:54.236186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:54.240500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c97f-58ae-459e-aa4b-94eb89cc8006 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.241676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 332/659] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:54.242740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:54.265897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:54.287546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf9cb2-c305-4a02-b646-20175b928f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.288211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf9cb2-c305-4a02-b646-20175b928f0a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.288493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf9cb2-c305-4a02-b646-20175b928f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.288769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf9cb2-c305-4a02-b646-20175b928f0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.367216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf9cb2-c305-4a02-b646-20175b928f0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:54.368917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf9cb2-c305-4a02-b646-20175b928f0a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:54.369636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c01ae1-a5f5-48df-81b5-6b0f2cc6d573 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.397318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:54.405457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.406135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.406415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.406689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.407529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:54.407835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:54.409284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:54.432226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 333/660] 10.222.0.66 () {56 vars in 1139 bytes} [Thu Jun 16 17:09:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:54.432226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:54.488956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 328/661] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:54.488956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af2fb7-d538-4365-a1be-242889230ab4 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.488956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af2fb7-d538-4365-a1be-242889230ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.488956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af2fb7-d538-4365-a1be-242889230ab4 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.494058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 329/662] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:54.494885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af2fb7-d538-4365-a1be-242889230ab4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:54.516331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:54.540078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598f65c-5ebc-4c17-b390-e48139cd7a2d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.577489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.578152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.578433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.578706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.579428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:54.579729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:54.581207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:54.598198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 334/663] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:54.598198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:54.612664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af2fb7-d538-4365-a1be-242889230ab4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.639870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:54.649849 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646231a-42c6-42df-9314-8493537b0903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.650508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646231a-42c6-42df-9314-8493537b0903 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.650801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646231a-42c6-42df-9314-8493537b0903 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.651078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2646231a-42c6-42df-9314-8493537b0903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.707118 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a68836b-8c0e-4bd8-8cce-1f3606a07718 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.721966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646231a-42c6-42df-9314-8493537b0903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:54.723942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2646231a-42c6-42df-9314-8493537b0903 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:54.743413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7159247-45c8-4edd-aa1a-8efb3ce9779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.744241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7159247-45c8-4edd-aa1a-8efb3ce9779c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.744527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7159247-45c8-4edd-aa1a-8efb3ce9779c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.744801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7159247-45c8-4edd-aa1a-8efb3ce9779c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.747769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:54.758280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7159247-45c8-4edd-aa1a-8efb3ce9779c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:54.759729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7159247-45c8-4edd-aa1a-8efb3ce9779c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b7159247-45c8-4edd-aa1a-8efb3ce9779c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:54.778065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:54.779032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:54.779961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:54.779961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:54.779961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:54.779961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:54.779961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 335/664] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:54.780786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66e75-3d19-46de-8564-d822e6f67097 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.780954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66e75-3d19-46de-8564-d822e6f67097 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:54.781367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca66e75-3d19-46de-8564-d822e6f67097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:54.797866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-bca66e75-3d19-46de-8564-d822e6f67097 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:54.799046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bca66e75-3d19-46de-8564-d822e6f67097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:54.833034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 330/665] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:54.834064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646231a-42c6-42df-9314-8493537b0903 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:54.864113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:54.875930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 336/666] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:54.877650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca66e75-3d19-46de-8564-d822e6f67097 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:54.959226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2646231a-42c6-42df-9314-8493537b0903 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.987976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca66e75-3d19-46de-8564-d822e6f67097 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:54.998451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:54.999484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:54.999786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.000196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.006275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:55.007269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.008711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:55.022512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:55.023436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:55.024376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 331/667] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:55.024873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.031905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.032586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.032870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.033148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.033904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:55.034217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.036049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.055817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 337/668] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:55.055817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.109007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:55.124266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a331e82-8978-46ed-82b9-6a8b6dd1a32c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.159665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.160468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.160751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.161053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.171936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.173657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.177352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f631214-2e0c-407b-b020-39c158bdaa6b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:55.191426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:55.192559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:55.193786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:55.193786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:55.193786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:55.193786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:55.194464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 332/669] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:55.195345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.221856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996e0b0-8cf3-46c8-8d0f-c1c4aa89e565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.222799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996e0b0-8cf3-46c8-8d0f-c1c4aa89e565 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.223083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996e0b0-8cf3-46c8-8d0f-c1c4aa89e565 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.223356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996e0b0-8cf3-46c8-8d0f-c1c4aa89e565 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.237094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996e0b0-8cf3-46c8-8d0f-c1c4aa89e565 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-4996e0b0-8cf3-46c8-8d0f-c1c4aa89e565 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:55.245210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:55.246251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:09:55.247112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:55.251550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 338/670] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:55.252333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996e0b0-8cf3-46c8-8d0f-c1c4aa89e565 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.297055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:55.313922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9547c0-8445-497a-a912-a2232f13b1e7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.352082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.352745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.353031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.353343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.354135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:55.354444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.355903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.360383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996e0b0-8cf3-46c8-8d0f-c1c4aa89e565 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.365300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:55.375666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 333/671] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:55.376398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.398233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.399031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.399304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.399574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.406968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.408496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:55.422424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:55.423356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:55.424253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 339/672] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:55.425529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.488409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25d081-40cb-49fc-b9c6-d97b28fc8808 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.524151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.524883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.525297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.525679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.526426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:55.526820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.528268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.531042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:55.535618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f6ae-2309-49cd-8fe4-4279378ca773 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.541013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 334/673] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:55.541644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.575424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.576088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.576368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.576638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.577413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:55.577714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.579075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.587837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 340/674] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:55.589050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.607808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:55.653955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015275c6-3676-431e-85da-1f6f951e643f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.691138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.691815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.692102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.692376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.693095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1375f4e-0fbd-46cf-8ebd-0c3aebfaac97 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.693348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=d6179dc2bd1d4ff2a04a8362d63e7b7c) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:55.693661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.695113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.701223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 335/675] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/role_assignments?user.id=d6179dc2bd1d4ff2a04a8362d63e7b7c&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:55.701997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.752342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.753819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.754332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.755080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.756917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:55.757781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.760351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.781842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 341/676] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:55.783137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.814873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:55.825055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2119-a0ef-40ee-a764-bc4db969b16b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.859719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:55.887732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.891067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.891067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.891067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.899451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.902150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.915099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955353c6-9795-46cf-aa16-160dca13756e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:55.919140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:55.920086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:55.921482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 336/677] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:55.922012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:55.963482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:55.964282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:55.964569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:55.964833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:55.975629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:55.977251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:55.995963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:55.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:55.998543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:55.998543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:55.998543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:55.998543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:55.998543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 342/678] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:55.998543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:56.058292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f5391-6e8c-41bd-8daf-ffec2ab80fef None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.075550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:56.108517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d332b0c-6e58-407a-ab8f-fcfb6dea1612 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.109282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.110480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.111884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.112371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.113448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=db842931b2394e8684012057347080af) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:56.114043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:56.115904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:56.125545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 337/679] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/role_assignments?user.id=db842931b2394e8684012057347080af&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:56.126988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:56.146994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.147682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.147962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.148292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.149124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:56.149479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:56.150984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:56.164816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 343/680] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:56.170908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:56.209051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:56.311990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997f8bbb-6800-4971-92d3-9a8fcff1938f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.351202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b935b-8355-4ee1-9a77-1ccc5ff79cfc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.353101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cd667-60a2-4d7e-af6f-a733f454daad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.353825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cd667-60a2-4d7e-af6f-a733f454daad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.354112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cd667-60a2-4d7e-af6f-a733f454daad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.354398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cd667-60a2-4d7e-af6f-a733f454daad None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.355204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cd667-60a2-4d7e-af6f-a733f454daad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:56.355464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cd667-60a2-4d7e-af6f-a733f454daad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:56.356918 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cd667-60a2-4d7e-af6f-a733f454daad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:56.371214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 338/681] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:56.372923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd9d3-30c7-41e2-a09e-962ccc3e79bb None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.373578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd9d3-30c7-41e2-a09e-962ccc3e79bb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.374060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbd9d3-30c7-41e2-a09e-962ccc3e79bb None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.375811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 339/682] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:56.376779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbd9d3-30c7-41e2-a09e-962ccc3e79bb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:56.384687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:56.391800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaf046-fc6a-4c3f-8376-b63a2e350701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.392498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf046-fc6a-4c3f-8376-b63a2e350701 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.392777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf046-fc6a-4c3f-8376-b63a2e350701 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.393049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf046-fc6a-4c3f-8376-b63a2e350701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.394259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaf046-fc6a-4c3f-8376-b63a2e350701 None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=72fb6996e1974060b162070f8da365aa) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:56.394585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaf046-fc6a-4c3f-8376-b63a2e350701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:56.396137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaf046-fc6a-4c3f-8376-b63a2e350701 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:56.403014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 344/683] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:56 2022] GET /identity/v3/role_assignments?user.id=72fb6996e1974060b162070f8da365aa&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:56.403704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a9f2b-a34c-430c-b7eb-c35cda342ff8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.403984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a9f2b-a34c-430c-b7eb-c35cda342ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.404263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a9f2b-a34c-430c-b7eb-c35cda342ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.437372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7b5a9f2b-a34c-430c-b7eb-c35cda342ff8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:56.438764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7b5a9f2b-a34c-430c-b7eb-c35cda342ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:56.528256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecbd9d3-30c7-41e2-a09e-962ccc3e79bb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.537086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:56.556360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 345/684] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:56.575528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.576301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.576585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.576863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.586106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:56.587514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a9f2b-a34c-430c-b7eb-c35cda342ff8 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:56.587647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:56.604148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:56.605235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:56.606206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 340/685] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:09:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:56.614332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:56.724138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a9f2b-a34c-430c-b7eb-c35cda342ff8 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.736296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb52872-1ecc-4c02-b124-d34a065acdbe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.761340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:56.765195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0a83-cb16-4e34-a997-04633a43c243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.766047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0a83-cb16-4e34-a997-04633a43c243 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.766408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0a83-cb16-4e34-a997-04633a43c243 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.766607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb0a83-cb16-4e34-a997-04633a43c243 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.777108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.778206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.778679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abb0a83-cb16-4e34-a997-04633a43c243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:09:56.778907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.779370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.780430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:56.780945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:56.782652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9abb0a83-cb16-4e34-a997-04633a43c243 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:09:56.788521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:56.789571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:56.790447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 346/686] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:09:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:56.795959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0a83-cb16-4e34-a997-04633a43c243 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:56.806989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 341/687] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Jun 16 17:09:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:56.812447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:56.840874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:56.910326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb0a83-cb16-4e34-a997-04633a43c243 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.924177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b144a5-5d2a-43f2-a9fa-98546128671d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:56.948476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.949251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.949482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.949763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.950544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:56.950865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:56.952487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:56.962952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 347/688] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:09:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:56.964531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cbcee-27ab-4788-96cc-40a0d760c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:56.966977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cbcee-27ab-4788-96cc-40a0d760c39a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:56.966977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cbcee-27ab-4788-96cc-40a0d760c39a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:56.966977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4cbcee-27ab-4788-96cc-40a0d760c39a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:56.978800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:57.045233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cbcee-27ab-4788-96cc-40a0d760c39a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:57.045233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4cbcee-27ab-4788-96cc-40a0d760c39a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:57.066092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:57.069064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:57.168599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c82690-d2cc-40f0-b437-f34d494b88ff None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:57.190999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 342/689] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:57.194457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cbcee-27ab-4788-96cc-40a0d760c39a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:57.236394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:57.237647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:57.238144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:57.238612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:57.262211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:57.264053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:57.293911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:57.294888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:57.295825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:57.295825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:57.295825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:57.295825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:57.303150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 348/690] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:57.307931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:57.389583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4cbcee-27ab-4788-96cc-40a0d760c39a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:57.429471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:57.453026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:57.454214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:57.454717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:57.455200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:57.476693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:57.478537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:57.507204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcef7fd-f109-4f26-9d54-3420111d619b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:57.509103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:57.509431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:57.510316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:57.511480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:57.511480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:57.511480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:57.511480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:57.511480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 343/691] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:57.520071 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:57.572787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:57.573633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:57.573921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:57.574195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:57.574926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:57.575228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:57.576606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:57.603734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 349/692] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:57.608872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:57.690181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac3684-62d3-4928-82f5-7c52b60ccc35 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:57.755698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:57.756554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:57.756834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:57.757107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:57.757854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:57.758155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:57.759520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:57.784284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 344/693] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:57.785221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:57.788084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc34bb0-aa4f-4650-8dc1-7923d0f73e29 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:57.847450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:57.848663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:57.851334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:57.851615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:57.851887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:57.866036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:57.867501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:09:57.892533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:57.893595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:09:57.894553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:57.895095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 350/694] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:09:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:57.904009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:57.947463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:57.970230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e283e-1720-41cb-8732-812dbd6e8aca None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:58.041081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1b83e-33e0-43f3-b839-99dc76c27e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:58.041910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b83e-33e0-43f3-b839-99dc76c27e55 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.042188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b83e-33e0-43f3-b839-99dc76c27e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.042460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b83e-33e0-43f3-b839-99dc76c27e55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.052949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1b83e-33e0-43f3-b839-99dc76c27e55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:58.054428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1b83e-33e0-43f3-b839-99dc76c27e55 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b7e1b83e-33e0-43f3-b839-99dc76c27e55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:58.072201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:58.073600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:58.074509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 345/695] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:58.074994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae33c31-4c21-4019-aac7-b96d0b91ae34 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.074994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae33c31-4c21-4019-aac7-b96d0b91ae34 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.075671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae33c31-4c21-4019-aac7-b96d0b91ae34 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.076646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 346/696] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:58.078934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae33c31-4c21-4019-aac7-b96d0b91ae34 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:58.083583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaaed62-4ef8-4344-a671-f70b0cba6575 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:58.121243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f6167-984a-4755-a836-07a0b3947745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:58.121920 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f6167-984a-4755-a836-07a0b3947745 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.122200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f6167-984a-4755-a836-07a0b3947745 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.122474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f6167-984a-4755-a836-07a0b3947745 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.123311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f6167-984a-4755-a836-07a0b3947745 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:58.123619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f6167-984a-4755-a836-07a0b3947745 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:58.125155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f6167-984a-4755-a836-07a0b3947745 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:58.144459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 351/697] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:09:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:58.150769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782b19-c6fe-4dd9-b153-a538ccbf7066 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.151079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782b19-c6fe-4dd9-b153-a538ccbf7066 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.151326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5782b19-c6fe-4dd9-b153-a538ccbf7066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.168579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f5782b19-c6fe-4dd9-b153-a538ccbf7066 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:58.169906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f5782b19-c6fe-4dd9-b153-a538ccbf7066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:58.192647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:58.195519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae33c31-4c21-4019-aac7-b96d0b91ae34 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:58.233736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:58.234389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.234680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.234961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.235806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:58.236120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:58.237789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:58.249500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 352/698] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:58.250628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5782b19-c6fe-4dd9-b153-a538ccbf7066 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:58.256162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 347/699] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:58.261465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:58.262966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:58.354527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5782b19-c6fe-4dd9-b153-a538ccbf7066 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:58.398988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51d20b-8d24-43ae-a4dc-850445a36b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:58.399905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51d20b-8d24-43ae-a4dc-850445a36b84 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.400185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51d20b-8d24-43ae-a4dc-850445a36b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.400456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51d20b-8d24-43ae-a4dc-850445a36b84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.407759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154264a-92e3-45ff-962e-58a3ba4504be None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:58.444398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:58.445270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.445700 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.446077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.447012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=0f15957f640a411eb4ac8861454f2b97) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:58.447448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:58.449050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:58.455200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 348/700] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:58 2022] GET /identity/v3/role_assignments?user.id=0f15957f640a411eb4ac8861454f2b97&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:09:58.456717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:58.509660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:58.513424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51d20b-8d24-43ae-a4dc-850445a36b84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:58.515031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51d20b-8d24-43ae-a4dc-850445a36b84 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:58.536334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:58.663241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be62d0d-3c03-4313-8b03-c3a8422b12e2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:58.712522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 353/701] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:09:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 17:09:58.720499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51d20b-8d24-43ae-a4dc-850445a36b84 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:58.729627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:58.731319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.731777 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.732247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.733603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:58.735943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:58.756049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:58.757133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:09:58.758014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:58.762077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 349/702] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Jun 16 17:09:58 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:58.767494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:58.850622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51d20b-8d24-43ae-a4dc-850445a36b84 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:58.862886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:58.880714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:58.901330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1000db-3746-4f33-b89b-9c51a67d2cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:58.902139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1000db-3746-4f33-b89b-9c51a67d2cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.902249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bcc034-b8f8-49e7-bdf5-7160b39a64ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:58.902428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1000db-3746-4f33-b89b-9c51a67d2cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.902710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1000db-3746-4f33-b89b-9c51a67d2cd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.916465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1000db-3746-4f33-b89b-9c51a67d2cd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:58.919066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1000db-3746-4f33-b89b-9c51a67d2cd6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:58.940294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc45bf-87ed-4438-86ca-469ab7210a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:58.941311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc45bf-87ed-4438-86ca-469ab7210a1c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7f1000db-3746-4f33-b89b-9c51a67d2cd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:58.941444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:58.942271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:58.943278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:58.943278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:58.943278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:58.943278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:58.943278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 354/703] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:09:58.944346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc45bf-87ed-4438-86ca-469ab7210a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.944829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245e535-2357-46d9-922c-4f4fc00a7f22 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.945029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc45bf-87ed-4438-86ca-469ab7210a1c None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.945196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245e535-2357-46d9-922c-4f4fc00a7f22 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.945584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245e535-2357-46d9-922c-4f4fc00a7f22 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.946301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc45bf-87ed-4438-86ca-469ab7210a1c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:58.946801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc45bf-87ed-4438-86ca-469ab7210a1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:58.948641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc45bf-87ed-4438-86ca-469ab7210a1c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:58.951333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 355/704] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:58.952749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245e535-2357-46d9-922c-4f4fc00a7f22 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:58.955973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 350/705] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Jun 16 17:09:58 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:58.957599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb178-04e0-4294-8514-b7c373ae8065 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:58.958198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb178-04e0-4294-8514-b7c373ae8065 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:58.958689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9bb178-04e0-4294-8514-b7c373ae8065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:58.981127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1c9bb178-04e0-4294-8514-b7c373ae8065 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:58.982738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1c9bb178-04e0-4294-8514-b7c373ae8065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:59.064339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 351/706] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:59.065631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bb178-04e0-4294-8514-b7c373ae8065 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.066074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245e535-2357-46d9-922c-4f4fc00a7f22 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.113226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.114006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.114166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.114564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.115478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:59.115791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.117242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.133647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 356/707] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:59.134621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.154435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:59.157933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:59.185681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9bb178-04e0-4294-8514-b7c373ae8065 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.225776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.226788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.227266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.227724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.228896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:59.229431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.231073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.237297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 352/708] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Jun 16 17:09:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:59.238115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.258928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862d7a7-c4ac-4d45-b78a-3844eaa9b174 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.308442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c218471-afec-41c4-b46d-2083c6dd8b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.309315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c218471-afec-41c4-b46d-2083c6dd8b0f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.309602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c218471-afec-41c4-b46d-2083c6dd8b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.309878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c218471-afec-41c4-b46d-2083c6dd8b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.331077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c218471-afec-41c4-b46d-2083c6dd8b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.332505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c218471-afec-41c4-b46d-2083c6dd8b0f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9c218471-afec-41c4-b46d-2083c6dd8b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:09:59.358096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:59.359258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:59.360187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:09:59.360187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:09:59.360187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:09:59.360187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:59.360187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 357/709] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:59.363058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8cf77-30c3-490e-8544-a0c68e71129f None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.363058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8cf77-30c3-490e-8544-a0c68e71129f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.363058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8cf77-30c3-490e-8544-a0c68e71129f None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.363353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075ccad-d8ce-4159-a1de-35d24b3bc7aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.363444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 358/710] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:09:59.366072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8cf77-30c3-490e-8544-a0c68e71129f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.413374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8494cb-7b58-41b0-a314-a085fc364a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.414304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8494cb-7b58-41b0-a314-a085fc364a9d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.414660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8494cb-7b58-41b0-a314-a085fc364a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.415008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8494cb-7b58-41b0-a314-a085fc364a9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.424469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8494cb-7b58-41b0-a314-a085fc364a9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.425958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8494cb-7b58-41b0-a314-a085fc364a9d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-dc8494cb-7b58-41b0-a314-a085fc364a9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:09:59.441445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:59.442419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:59.443773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:09:59.443773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:09:59.443773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:09:59.443773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:09:59.443773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:09:59.443773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:09:59.443773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:09:59.443773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:09:59.445319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 353/711] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:59.447456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61d65-0fd7-465e-840e-b34f4950d853 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.448470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61d65-0fd7-465e-840e-b34f4950d853 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.448921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d61d65-0fd7-465e-840e-b34f4950d853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.460273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:59.462874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:59.483421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a9d61d65-0fd7-465e-840e-b34f4950d853 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:09:59.486035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9d61d65-0fd7-465e-840e-b34f4950d853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:09:59.498360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8cf77-30c3-490e-8544-a0c68e71129f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.536377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.537235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.537619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.537995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.538944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:59.539348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.540834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.554524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 359/712] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:59.555498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.578764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 354/713] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:09:59.579649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d61d65-0fd7-465e-840e-b34f4950d853 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.667922 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98037ad-1c3d-46b2-b565-efb3057c7c3b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.696868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d61d65-0fd7-465e-840e-b34f4950d853 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.699045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:59.705582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.706375 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.707105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.707226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.708144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:59.708575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.710709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.728799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 360/714] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:09:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:09:59.729898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.732107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:09:59.738577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c384-5e57-4faf-a02c-8fb4d70579cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.739267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c384-5e57-4faf-a02c-8fb4d70579cf None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.739590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c384-5e57-4faf-a02c-8fb4d70579cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.739980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c384-5e57-4faf-a02c-8fb4d70579cf None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.740745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016c384-5e57-4faf-a02c-8fb4d70579cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.742257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016c384-5e57-4faf-a02c-8fb4d70579cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.766077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 355/715] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:09:59 2022] POST /identity/v3/users => generated 254 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:09:59.767184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c384-5e57-4faf-a02c-8fb4d70579cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.844811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420d3c-fa68-4fde-8bda-5845174ab226 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.882164 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.883473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.884008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.884507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] PATH_INFO: `/v3/services/network` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.885436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.887014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c384-5e57-4faf-a02c-8fb4d70579cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:09:59.887365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:09:59.903153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:09:59.904283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:09:59.905139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:09:59.908230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 361/716] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:09:59 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:09:59.914612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:09:59.941167 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:09:59.941881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:09:59.942220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:09:59.942536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:09:59.943401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=e3241452a88a40da9c9b8fa79c4fc9c0) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:09:59.943715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:09:59.945481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:09:59.959550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 356/717] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:09:59 2022] GET /identity/v3/role_assignments?user.id=e3241452a88a40da9c9b8fa79c4fc9c0&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:00.010204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:00.070248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13866de5-c8af-48ec-9c40-39897248cfef None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:00.108804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2402715-51e6-48cc-b948-f1dc0eb15460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:00.109516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402715-51e6-48cc-b948-f1dc0eb15460 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:00.109800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402715-51e6-48cc-b948-f1dc0eb15460 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:00.110076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402715-51e6-48cc-b948-f1dc0eb15460 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:00.110803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2402715-51e6-48cc-b948-f1dc0eb15460 None admin] RBAC: Adding query filter params (name=network) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:00.111110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2402715-51e6-48cc-b948-f1dc0eb15460 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:00.112519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2402715-51e6-48cc-b948-f1dc0eb15460 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:00.118443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 362/718] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:09:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:00.126424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:00.190758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:00.242254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0dcc0-66a9-4d59-ae1f-5408cb58c156 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:00.278944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6e346-7818-44d2-9c9d-ab0d30ce0a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:00.279616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e346-7818-44d2-9c9d-ab0d30ce0a79 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:00.279895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e346-7818-44d2-9c9d-ab0d30ce0a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:00.280166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e346-7818-44d2-9c9d-ab0d30ce0a79 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:00.280872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6e346-7818-44d2-9c9d-ab0d30ce0a79 None admin] RBAC: Adding query filter params (type=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:00.281199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6e346-7818-44d2-9c9d-ab0d30ce0a79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:00.282620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6e346-7818-44d2-9c9d-ab0d30ce0a79 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:00.287904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 357/719] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:01.129134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d40bb8-560d-46c3-b35f-505bd47268b0 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:01.130305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d40bb8-560d-46c3-b35f-505bd47268b0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:01.131009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d40bb8-560d-46c3-b35f-505bd47268b0 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:01.132821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 363/720] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:01.138533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c2e90-229f-4e21-84d9-9ce4eb4866d6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:01.138826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c2e90-229f-4e21-84d9-9ce4eb4866d6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:01.139105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c2e90-229f-4e21-84d9-9ce4eb4866d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:01.165866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-966c2e90-229f-4e21-84d9-9ce4eb4866d6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:01.167062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-966c2e90-229f-4e21-84d9-9ce4eb4866d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:01.262917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 358/721] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:10:01.303289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d40bb8-560d-46c3-b35f-505bd47268b0 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:01.434800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d40bb8-560d-46c3-b35f-505bd47268b0 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:01.459357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:01.484847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df920209-1356-40f9-8daf-4611136fae96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:01.490637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df920209-1356-40f9-8daf-4611136fae96 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:01.490927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df920209-1356-40f9-8daf-4611136fae96 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:01.491207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df920209-1356-40f9-8daf-4611136fae96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:01.514802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df920209-1356-40f9-8daf-4611136fae96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-df920209-1356-40f9-8daf-4611136fae96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:01.522733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:01.523767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:01.524624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 364/722] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:01.534349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c2e90-229f-4e21-84d9-9ce4eb4866d6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:01.613295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:01.685528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c2e90-229f-4e21-84d9-9ce4eb4866d6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:01.737088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493bfcd-52ec-47ce-9b52-5ed43cae5d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:01.737788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493bfcd-52ec-47ce-9b52-5ed43cae5d05 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:01.738067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493bfcd-52ec-47ce-9b52-5ed43cae5d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:01.738341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493bfcd-52ec-47ce-9b52-5ed43cae5d05 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:01.739100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493bfcd-52ec-47ce-9b52-5ed43cae5d05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:01.739543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493bfcd-52ec-47ce-9b52-5ed43cae5d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:01.741051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1493bfcd-52ec-47ce-9b52-5ed43cae5d05 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:01.750119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 359/723] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:01.756208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df920209-1356-40f9-8daf-4611136fae96 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:01.892395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df920209-1356-40f9-8daf-4611136fae96 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:01.895314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce2940-f681-4ad1-8bf6-cc9080a00041 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:01.895972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce2940-f681-4ad1-8bf6-cc9080a00041 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:01.896492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce2940-f681-4ad1-8bf6-cc9080a00041 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:01.898325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 360/724] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:01.908337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf4a1-440b-4dc0-9ae6-5ca838d495e4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:01.908623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf4a1-440b-4dc0-9ae6-5ca838d495e4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:01.908902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bf4a1-440b-4dc0-9ae6-5ca838d495e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:01.930957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:01.931756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:01.932084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:01.932364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:01.944546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:01.945963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:01.947935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-268bf4a1-440b-4dc0-9ae6-5ca838d495e4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:01.949140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-268bf4a1-440b-4dc0-9ae6-5ca838d495e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:01.966399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:01.967391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:01.968294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:01.968294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:01.968294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:01.968294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:01.968294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 365/725] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:01.974584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:02.038277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:02.068815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 361/726] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:10:02.110273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bf4a1-440b-4dc0-9ae6-5ca838d495e4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:02.128226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31307ea5-d617-4cca-b11b-1ccfacb226a6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:02.167946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:02.168729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.173206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.173206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.173206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:02.173206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:02.173206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:02.182831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:02.199390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 366/727] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:02.206824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:02.310717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268bf4a1-440b-4dc0-9ae6-5ca838d495e4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:02.372945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909372d-c787-4ef4-b5c4-ba26308d8768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:02.373670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909372d-c787-4ef4-b5c4-ba26308d8768 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.373957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909372d-c787-4ef4-b5c4-ba26308d8768 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.374802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909372d-c787-4ef4-b5c4-ba26308d8768 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.375597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909372d-c787-4ef4-b5c4-ba26308d8768 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:02.377023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909372d-c787-4ef4-b5c4-ba26308d8768 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:02.386409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:02.403999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 362/728] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:10:02 2022] POST /identity/v3/services => generated 251 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:02.419755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20bb13-d0f8-49d5-ad64-c70d00016ed1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:02.426036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171efb31-d6c3-484a-a873-ff94f0f8526e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.426561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171efb31-d6c3-484a-a873-ff94f0f8526e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.426984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171efb31-d6c3-484a-a873-ff94f0f8526e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.428996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 363/729] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:02.436577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad7b2c-a8b0-4e87-95f6-8dd09554d96b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.437821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad7b2c-a8b0-4e87-95f6-8dd09554d96b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.438406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad7b2c-a8b0-4e87-95f6-8dd09554d96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.461100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ddad7b2c-a8b0-4e87-95f6-8dd09554d96b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:02.462948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ddad7b2c-a8b0-4e87-95f6-8dd09554d96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:02.522223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:02.523161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.523450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.523737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.546248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:02.547764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:02.549362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 364/730] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:02.569143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:02.570391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:02.571500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 367/731] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:02.576733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad7b2c-a8b0-4e87-95f6-8dd09554d96b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:02.591596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:02.617616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:02.703930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad7b2c-a8b0-4e87-95f6-8dd09554d96b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:02.730346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a297f2-6ad3-4f22-a580-2ef717a00265 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:02.749359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d17b9-3fd5-4752-a327-bf9ddaebc7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:02.750395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d17b9-3fd5-4752-a327-bf9ddaebc7ac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.750759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d17b9-3fd5-4752-a327-bf9ddaebc7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.751245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d17b9-3fd5-4752-a327-bf9ddaebc7ac None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.752484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d17b9-3fd5-4752-a327-bf9ddaebc7ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:02.752865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d17b9-3fd5-4752-a327-bf9ddaebc7ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:02.754469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d17b9-3fd5-4752-a327-bf9ddaebc7ac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:02.775398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32f099-09f6-44f4-a0cc-22de6eb7c653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:02.776300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32f099-09f6-44f4-a0cc-22de6eb7c653 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.776647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32f099-09f6-44f4-a0cc-22de6eb7c653 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.776986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32f099-09f6-44f4-a0cc-22de6eb7c653 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.777688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 365/732] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Jun 16 17:10:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:02.779176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac91c8-f979-4df1-a6d7-caff5d459f60 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.779534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac91c8-f979-4df1-a6d7-caff5d459f60 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.779878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac91c8-f979-4df1-a6d7-caff5d459f60 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.781505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 366/733] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:02.783806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac91c8-f979-4df1-a6d7-caff5d459f60 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:02.794475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32f099-09f6-44f4-a0cc-22de6eb7c653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3f32f099-09f6-44f4-a0cc-22de6eb7c653 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:02.808846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:02.809906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:02.810773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:02.811463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 368/734] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:02.817485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84fa8-e864-40c6-964d-94791bb20ada None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.817835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84fa8-e864-40c6-964d-94791bb20ada None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.818119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d84fa8-e864-40c6-964d-94791bb20ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:02.840168 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-51d84fa8-e864-40c6-964d-94791bb20ada None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:02.842185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-51d84fa8-e864-40c6-964d-94791bb20ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:02.857537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:02.900357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:02.920223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac91c8-f979-4df1-a6d7-caff5d459f60 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:02.929263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 369/735] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:02 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:10:02.930426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d84fa8-e864-40c6-964d-94791bb20ada None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:02.978914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ffab1-cc9a-48af-95d1-9dcad8fada02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:02.979625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ffab1-cc9a-48af-95d1-9dcad8fada02 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:02.979910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ffab1-cc9a-48af-95d1-9dcad8fada02 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:02.980693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ffab1-cc9a-48af-95d1-9dcad8fada02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.063643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d84fa8-e864-40c6-964d-94791bb20ada None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:03.073740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ffab1-cc9a-48af-95d1-9dcad8fada02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:03.075283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ffab1-cc9a-48af-95d1-9dcad8fada02 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:03.109713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:03.110422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.110732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.110970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.111837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:03.112039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:03.114095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:03.121440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:03.126399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 370/736] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:03.127517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.178595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:03.200881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 367/737] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:02 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 17:10:03.200881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd7183-edd6-4492-845f-4d1dc59ec8e6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.200881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd7183-edd6-4492-845f-4d1dc59ec8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.200881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd7183-edd6-4492-845f-4d1dc59ec8e6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.200881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 368/738] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:03.200881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd7183-edd6-4492-845f-4d1dc59ec8e6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.235839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f86e9-82cb-4203-aca5-c48f8bbe3ab1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:03.271632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f986a-7dee-419f-8b47-15d63d1b26c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:03.272477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f986a-7dee-419f-8b47-15d63d1b26c4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.272756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f986a-7dee-419f-8b47-15d63d1b26c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.273028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f986a-7dee-419f-8b47-15d63d1b26c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.284100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f986a-7dee-419f-8b47-15d63d1b26c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-042f986a-7dee-419f-8b47-15d63d1b26c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:03.292462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:03.293451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:03.294295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 371/739] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:10:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:03.295565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f986a-7dee-419f-8b47-15d63d1b26c4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.311786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd7183-edd6-4492-845f-4d1dc59ec8e6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:03.348756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f5935-2d40-453b-97af-c457f884700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:03.349593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f5935-2d40-453b-97af-c457f884700f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.349869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f5935-2d40-453b-97af-c457f884700f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.350138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f5935-2d40-453b-97af-c457f884700f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.357998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:03.362503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f5935-2d40-453b-97af-c457f884700f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:03.364258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f5935-2d40-453b-97af-c457f884700f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-330f5935-2d40-453b-97af-c457f884700f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:03.383354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:03.384301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:03.385259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:03.385259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:03.385259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:03.385259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:03.385259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 369/740] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:03.386075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4a5d3-1dc3-4a90-bfba-6036b4467cb1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.386075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4a5d3-1dc3-4a90-bfba-6036b4467cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.386358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4a5d3-1dc3-4a90-bfba-6036b4467cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.402435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1db4a5d3-1dc3-4a90-bfba-6036b4467cb1 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:03.403639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1db4a5d3-1dc3-4a90-bfba-6036b4467cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:03.410073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f986a-7dee-419f-8b47-15d63d1b26c4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:03.434290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:03.446970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:03.447818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.448058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.448334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.459372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:03.461143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:03.483964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 370/741] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:03 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 17:10:03.483964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4a5d3-1dc3-4a90-bfba-6036b4467cb1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:03.484207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:03.485057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:03.486035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:03.486035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:03.486035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:03.486035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:03.486035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 372/742] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:03.486035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.586568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4a5d3-1dc3-4a90-bfba-6036b4467cb1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:03.603064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36027528-7ad6-41c9-a74f-dd3a1bbda24a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:03.616721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:03.623911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:03.624604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.624885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.625157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.625960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:03.626826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:03.628235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:03.642082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 371/743] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:10:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:03.642082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.645573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:03.646315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.646594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.646868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.647597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:03.647902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:03.649285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:03.666443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 373/744] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:03.666443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.680697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:03.754945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba61fe-02fa-4259-8e98-53808df2a957 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:03.774556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71534dc9-af8d-40c1-9056-4d877743366b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:03.792200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:03.792875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.793193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.793472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.794215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:03.794527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:03.795918 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:03.808416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 372/745] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:03.809234 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.815008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf71759-3f10-43ca-a4de-fff94e26b831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:03.815782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf71759-3f10-43ca-a4de-fff94e26b831 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:03.816100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf71759-3f10-43ca-a4de-fff94e26b831 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:03.816377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf71759-3f10-43ca-a4de-fff94e26b831 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:03.817114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf71759-3f10-43ca-a4de-fff94e26b831 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:03.818976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf71759-3f10-43ca-a4de-fff94e26b831 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:03.835165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 374/746] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:10:03 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:03.836269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf71759-3f10-43ca-a4de-fff94e26b831 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:03.903800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:03.932515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:03.944584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c4cd2-7568-4d98-955c-ba97760cead8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.001020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.001889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.002170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.002451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.005603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf71759-3f10-43ca-a4de-fff94e26b831 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.015130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.016616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:04.035514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:04.036644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:04.039607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:04.039607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:04.039607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:04.039607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:04.039607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 373/747] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:04.039607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.063240 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.063750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.064032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.064308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.074419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.075900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:04.091395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:04.092483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 375/748] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:10:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:04.095390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.168285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab812b-862e-4fe0-b9d2-be04ce9efb90 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.193391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:04.202937 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aecd10-13b6-47a4-9adc-bb3b6b6656e4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.206949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.208285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.208724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.209244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.217616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.219319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.221485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:04.236524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:04.237538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:04.238461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:04.238461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:04.238461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:04.238461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:04.238461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:04.238461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:04.238461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:04.238461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:04.241454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 374/749] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:04.243144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.244100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.244936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.245272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.245756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.246557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:04.246928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.248582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.266711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 376/750] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:04.266711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.364828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f238a3-99ff-48f0-aa5d-fae841daadd0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.369049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f322a-02cd-413f-83eb-b417374bdbfa None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.403940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.405021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.405507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.405614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.406098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.406254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.406642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.407659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:04.407883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:04.408256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.408512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.409914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.410324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.423518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 377/751] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:04.424509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.429924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 375/752] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:04.430984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.440762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:04.498315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:04.539000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337894e6-9f5f-474e-88b6-2d070eb25e32 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.546752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff36c37-fdf9-48a1-9cf4-2cbbbd8c675f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.576546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.577593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.577931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.578268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.586821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.587063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.587912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.588391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.588766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.589029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.590224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=0f15957f640a411eb4ac8861454f2b97) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:04.590834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.592557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.599496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 376/753] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/role_assignments?user.id=0f15957f640a411eb4ac8861454f2b97&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:04.600720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:04.605764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:04.606840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:04.607830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:04.612094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 378/754] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:04.613586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.695756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:04.742058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73f6bb-27d4-450d-8ac5-0f781eafe1b1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.749463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be86b818-104f-4ae2-94cb-543869ac878b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.787570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:04.788489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d00afe-a951-48d4-a05e-b984a38998ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.790081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d00afe-a951-48d4-a05e-b984a38998ef None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.790665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d00afe-a951-48d4-a05e-b984a38998ef None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.791283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d00afe-a951-48d4-a05e-b984a38998ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.791894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bfbfc-5ece-47af-893b-3ce75d31658f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.792831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bfbfc-5ece-47af-893b-3ce75d31658f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.793117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bfbfc-5ece-47af-893b-3ce75d31658f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.793429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bfbfc-5ece-47af-893b-3ce75d31658f None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.794711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bfbfc-5ece-47af-893b-3ce75d31658f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:04.795027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bfbfc-5ece-47af-893b-3ce75d31658f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.797095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bfbfc-5ece-47af-893b-3ce75d31658f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.805956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d00afe-a951-48d4-a05e-b984a38998ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:04.807437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d00afe-a951-48d4-a05e-b984a38998ef None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:04.819488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 379/755] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:04.820937 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db86fd1-d7e5-4896-967f-957bb04e75bc None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.822070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db86fd1-d7e5-4896-967f-957bb04e75bc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.822420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db86fd1-d7e5-4896-967f-957bb04e75bc None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.823938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 380/756] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:04.825022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db86fd1-d7e5-4896-967f-957bb04e75bc None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c7d00afe-a951-48d4-a05e-b984a38998ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:04.828126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:04.829318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:04.830433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:04.830433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:04.830433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:04.830433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:04.831861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 377/757] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:04.835745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad6b4-3d0b-4113-9af2-9033b5e9a8c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.835745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad6b4-3d0b-4113-9af2-9033b5e9a8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.835745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ad6b4-3d0b-4113-9af2-9033b5e9a8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:04.850791 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8d3ad6b4-3d0b-4113-9af2-9033b5e9a8c8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:04.852084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8d3ad6b4-3d0b-4113-9af2-9033b5e9a8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:04.924818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 378/758] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:04.925958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad6b4-3d0b-4113-9af2-9033b5e9a8c8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:04.949014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db86fd1-d7e5-4896-967f-957bb04e75bc None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:04.985028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:04.992669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71e20b-a2ca-41eb-b48b-ebf064819f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:04.993402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71e20b-a2ca-41eb-b48b-ebf064819f2e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:04.993686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71e20b-a2ca-41eb-b48b-ebf064819f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:04.994171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71e20b-a2ca-41eb-b48b-ebf064819f2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.036773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ad6b4-3d0b-4113-9af2-9033b5e9a8c8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.038924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:05.073627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71e20b-a2ca-41eb-b48b-ebf064819f2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.075448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71e20b-a2ca-41eb-b48b-ebf064819f2e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.082667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.083422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.083708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.083996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.085557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:05.085871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.087259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.105589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 379/759] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:05.107151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.191547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 381/760] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:04 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:05.192537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71e20b-a2ca-41eb-b48b-ebf064819f2e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.236139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:05.239715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d54dc-6f32-4945-ae6d-82d539688e0b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.276023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.277287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.277765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.278248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.279093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.281136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:05.297889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:05.298855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:10:05.299741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:05.302834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 380/761] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:05.303634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.313416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71e20b-a2ca-41eb-b48b-ebf064819f2e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.340194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:05.356425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.357631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.358063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.358438 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.367502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.369826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:05.387467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:05.388435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:05.389413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 382/762] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:05.390446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.436212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2232a2d-861c-4374-922d-dcb96b307ab8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.485560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.489282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.489282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.489282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.501717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.503385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.516792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:05.521789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:05.522869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:05.523976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:05.523976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:05.523976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:05.523976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:05.525325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 381/763] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:05.526197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.532905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cd17c3-950d-434d-a34e-c6262c24ef4f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.578509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.579694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.580475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.580921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.581801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:05.582256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.584927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.592329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 383/764] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:05.593639 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.613624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:05.649664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1e7fb-3e53-4064-84bc-1a014a5b293a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.695706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.697271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.697788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.698507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.699669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:05.700162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.702390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.709516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead36ae-66e6-4984-86c1-a5c57a631db3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.722216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 382/765] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:05.723355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.747352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.748182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.748459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.748852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.749707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:05.750087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.751604 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.766266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 384/766] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:05.767350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.785903 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:05.867381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990ce7e-817c-42b1-8f21-548434092e2e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.878268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8180f30-bcce-4a21-b2f5-0222654ac34e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:05.918143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.918813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.919091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.919365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.920071 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:05.920375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.921795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.922111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12843-4f09-4543-a3f6-2ec364ec76b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:05.923793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12843-4f09-4543-a3f6-2ec364ec76b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:05.924684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12843-4f09-4543-a3f6-2ec364ec76b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:05.925347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b12843-4f09-4543-a3f6-2ec364ec76b3 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/e3241452a88a40da9c9b8fa79c4fc9c0/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:05.928268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b12843-4f09-4543-a3f6-2ec364ec76b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=e3241452a88a40da9c9b8fa79c4fc9c0, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:05.929930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b12843-4f09-4543-a3f6-2ec364ec76b3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:05.932315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 383/767] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:05.933837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:05.937153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:05.949585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 385/768] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:10:05 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/e3241452a88a40da9c9b8fa79c4fc9c0/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:10:05.951082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12843-4f09-4543-a3f6-2ec364ec76b3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:06.118265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:06.142477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b12843-4f09-4543-a3f6-2ec364ec76b3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:06.145651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4bb9cc-74a0-47dc-93ab-a2c4c6a8e193 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:06.177749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:06.178424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.178775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.179182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.180049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] RBAC: Adding query filter params (interface=public, service_id=2c55406811794cd3a2c4128c631f5e7a, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:06.180429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:06.181887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:06.193447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 386/769] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/endpoints?service_id=2c55406811794cd3a2c4128c631f5e7a&interface=public®ion_id=RegionOne => generated 194 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:06.195623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:06.197638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:06.198797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.199279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.199828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.207595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:06.210966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:06.227912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:06.228905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:06.229872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:06.229872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:06.229872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:06.229872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:06.229872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:06.229872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:06.229872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:06.229872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:06.232509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 384/770] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:06.240470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:06.313210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d71b5-f77b-47d7-b917-b83f93b95162 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:06.343824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9a654-b948-431c-af53-8d84b3f21b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:06.344623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9a654-b948-431c-af53-8d84b3f21b51 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.344829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9a654-b948-431c-af53-8d84b3f21b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.345459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9a654-b948-431c-af53-8d84b3f21b51 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.346189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9a654-b948-431c-af53-8d84b3f21b51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:06.347777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9a654-b948-431c-af53-8d84b3f21b51 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:06.357758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 387/771] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:10:06 2022] GET /identity/v3/services => generated 772 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:06.381102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073599f-74f8-4cb1-b9a0-51c3d063bba9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.381464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073599f-74f8-4cb1-b9a0-51c3d063bba9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.381838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073599f-74f8-4cb1-b9a0-51c3d063bba9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.384974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 388/772] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:06.390521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0767-83e1-447f-96b0-dba906c675e7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.390927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0767-83e1-447f-96b0-dba906c675e7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.391148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:06.391386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c0767-83e1-447f-96b0-dba906c675e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.404532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:06.430528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-334c0767-83e1-447f-96b0-dba906c675e7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:06.431733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-334c0767-83e1-447f-96b0-dba906c675e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:06.440404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861d9f3-63dc-41f2-862e-f77f408f9f74 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:06.482379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:06.483580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.484067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.484544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.485671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:06.486189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:06.487953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:06.519661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 385/774] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:06.519885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 389/774] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:06.526185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:06.546147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c0767-83e1-447f-96b0-dba906c675e7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:06.696040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a4099-12d5-4cfa-b3ef-306cafcf58e9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:06.719429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c0767-83e1-447f-96b0-dba906c675e7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:06.723660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:06.737871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:06.738887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.739432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.740645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.741841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=ae4e4bbf4359400289b9fd2ba8778067) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:06.742408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:06.744092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:06.751007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 386/775] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:06 2022] GET /identity/v3/role_assignments?user.id=ae4e4bbf4359400289b9fd2ba8778067&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:06.757784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:06.761542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458e938-139e-4a85-be4b-cd800ec37646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:06.762376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458e938-139e-4a85-be4b-cd800ec37646 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.762659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458e938-139e-4a85-be4b-cd800ec37646 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.762933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458e938-139e-4a85-be4b-cd800ec37646 None admin] PATH_INFO: `/v3/services/network` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.763608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458e938-139e-4a85-be4b-cd800ec37646 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:06.764998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458e938-139e-4a85-be4b-cd800ec37646 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-4458e938-139e-4a85-be4b-cd800ec37646 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:06.777904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:06.779003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:06.779871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 390/776] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:06 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:06.787464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:06.892475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31154af-4ae9-4384-bc8e-cf0208d95712 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.892813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31154af-4ae9-4384-bc8e-cf0208d95712 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.893024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31154af-4ae9-4384-bc8e-cf0208d95712 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.894578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 391/777] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:06.900500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe91d89-8c70-4cf9-a0af-9da855aedd1e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.900836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe91d89-8c70-4cf9-a0af-9da855aedd1e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.901044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe91d89-8c70-4cf9-a0af-9da855aedd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.917416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae322d-2c9c-4547-8e89-e6312ec779ff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:06.926343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2fe91d89-8c70-4cf9-a0af-9da855aedd1e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:06.927581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2fe91d89-8c70-4cf9-a0af-9da855aedd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:06.956020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:06.957087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:06.957589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:06.958131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:06.959215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:06.960119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:06.961784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:06.967982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 387/778] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:06.977551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:07.048047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 392/779] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:07.056779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:07.088156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe91d89-8c70-4cf9-a0af-9da855aedd1e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:07.149458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f214fbc6-e68d-46c5-90d0-5e3a1cd6f3e1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:07.225059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:07.226247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:07.227585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:07.228728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:07.229605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] RBAC: Adding query filter params (type=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:07.229930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:07.231361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:07.237275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 388/780] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:07.246187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:07.264781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe91d89-8c70-4cf9-a0af-9da855aedd1e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:07.312385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:07.328929 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27578d-c379-4d05-9dad-bbe555b12080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:07.330132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27578d-c379-4d05-9dad-bbe555b12080 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:07.330605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27578d-c379-4d05-9dad-bbe555b12080 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:07.331050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27578d-c379-4d05-9dad-bbe555b12080 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:07.355795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27578d-c379-4d05-9dad-bbe555b12080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1a27578d-c379-4d05-9dad-bbe555b12080 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:07.366765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:07.367734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:07.368601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:07.369288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 393/781] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:07.377995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27578d-c379-4d05-9dad-bbe555b12080 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:07.403182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa953a-3218-40d0-b9b7-890b04efe74a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:07.419718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:07.457515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:07.458377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:07.458612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:07.458802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:07.459930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] RBAC: Adding query filter params (interface=public, service_id=504d4cbc6de84853924b4a75e265a6bf, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:07.460244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:07.461645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:07.492219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 389/782] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Jun 16 17:10:07 2022] GET /identity/v3/endpoints?service_id=504d4cbc6de84853924b4a75e265a6bf&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:07.498302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:07.583832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27578d-c379-4d05-9dad-bbe555b12080 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:07.647909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:07.648594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:07.648872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:07.649148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:07.649943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:07.650256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:07.651641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:07.668572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 394/783] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:07.670488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a75113-cd71-4619-bcf9-468e4a8628da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:07.678354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:07.727393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b96f2-e6f2-4fed-b488-8f056c243d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:07.728072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b96f2-e6f2-4fed-b488-8f056c243d20 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:07.728365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b96f2-e6f2-4fed-b488-8f056c243d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:07.728651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b96f2-e6f2-4fed-b488-8f056c243d20 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:07.729417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b96f2-e6f2-4fed-b488-8f056c243d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:07.730847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b96f2-e6f2-4fed-b488-8f056c243d20 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:07.736526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 390/784] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:10:07 2022] GET /identity/v3/services => generated 772 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:07.763515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:07.878522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faab29-eca7-40d9-b456-dd5fc16a2019 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:07.943228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:07.944634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:07.945125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:07.945648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:07.962147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:07.963971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:07.997553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:07.998631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:07.999557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:07.999557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:07.999557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:07.999557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:07.999557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 395/785] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:08.010395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b96f2-e6f2-4fed-b488-8f056c243d20 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:08.065814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:08.131853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b96f2-e6f2-4fed-b488-8f056c243d20 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:08.176705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a37415-1646-4616-8f27-fe7038ed7734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:08.177458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37415-1646-4616-8f27-fe7038ed7734 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:08.177705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37415-1646-4616-8f27-fe7038ed7734 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:08.177985 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a37415-1646-4616-8f27-fe7038ed7734 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:08.178722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a37415-1646-4616-8f27-fe7038ed7734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:08.179030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a37415-1646-4616-8f27-fe7038ed7734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:08.180416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a37415-1646-4616-8f27-fe7038ed7734 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:08.193025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 391/786] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:08.198757 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:08.328929 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:08.337476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a03b6-8bf4-41e0-a787-94235c7b3e8b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:08.369763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95797e7-7804-4da0-93f7-bbbbbad773f0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:08.370397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95797e7-7804-4da0-93f7-bbbbbad773f0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:08.370870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95797e7-7804-4da0-93f7-bbbbbad773f0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:08.372689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 392/787] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:08.378510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d07fcd-fcf9-4987-a5b4-e5e7fc496868 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:08.379032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d07fcd-fcf9-4987-a5b4-e5e7fc496868 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:08.379523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d07fcd-fcf9-4987-a5b4-e5e7fc496868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:08.394518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:08.395378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:08.395660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:08.395941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:08.402197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-16d07fcd-fcf9-4987-a5b4-e5e7fc496868 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:08.403390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-16d07fcd-fcf9-4987-a5b4-e5e7fc496868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:08.414796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:08.417135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:08.457882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:08.458974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:08.460285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 396/788] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:08.475369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:08.480261 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 393/789] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:08.503178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d07fcd-fcf9-4987-a5b4-e5e7fc496868 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:08.543668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:08.611011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d07fcd-fcf9-4987-a5b4-e5e7fc496868 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:08.647912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e838b-7cb4-454b-aba6-db472fb2ebc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:08.648786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e838b-7cb4-454b-aba6-db472fb2ebc8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:08.649075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e838b-7cb4-454b-aba6-db472fb2ebc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:08.649375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e838b-7cb4-454b-aba6-db472fb2ebc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:08.660128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e838b-7cb4-454b-aba6-db472fb2ebc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-634e838b-7cb4-454b-aba6-db472fb2ebc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:08.672184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:08.673154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:08.674089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 394/790] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:08.679672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e838b-7cb4-454b-aba6-db472fb2ebc8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:08.771320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13925b3c-6383-4e3d-918e-8ea450fa179f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:08.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e838b-7cb4-454b-aba6-db472fb2ebc8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:08.813555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:08.813758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:08.814639 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:08.814810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:08.815032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:08.815940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:08.816252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:08.817965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:08.851867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:08.852703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:08.852986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:08.853281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:08.854084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:08.854385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:08.855897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:08.856290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 397/791] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Jun 16 17:10:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:08.857160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:08.878185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 395/792] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:08.883393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:08.896691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:09.018828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412f146-5af3-467c-8052-455eaec8bd10 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.046949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489bb97-39f1-482d-8cf5-69be66960676 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.058747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:09.059567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.059849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.060125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.084387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:09.086627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:09.108677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:09.114771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.115287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.115755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:09.121367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:09.122273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:09.123206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:09.123206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:09.123206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:09.123206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:09.132705 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 398/793] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:09.137408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:09.139031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:09.141445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:09.174256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:09.175220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:09.176150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:09.176150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:09.176150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:09.176150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:09.186253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 396/794] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:09.191993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:09.208887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:09.290312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:09.324892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066df0c-6c46-474e-9a50-d2216e0ae900 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.344948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b35ff8-4cbc-417a-a34a-181d6f442dea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.368168 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de20298-ee0e-490f-8cf5-4b843d1f07cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:09.369018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de20298-ee0e-490f-8cf5-4b843d1f07cd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.369335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de20298-ee0e-490f-8cf5-4b843d1f07cd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.369618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de20298-ee0e-490f-8cf5-4b843d1f07cd None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.370349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de20298-ee0e-490f-8cf5-4b843d1f07cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:09.370660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de20298-ee0e-490f-8cf5-4b843d1f07cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:09.372105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de20298-ee0e-490f-8cf5-4b843d1f07cd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:09.381485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57baa1-8f22-400c-af6b-d35b03199b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:09.382121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57baa1-8f22-400c-af6b-d35b03199b57 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.382404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57baa1-8f22-400c-af6b-d35b03199b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.382681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57baa1-8f22-400c-af6b-d35b03199b57 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.383495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57baa1-8f22-400c-af6b-d35b03199b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:09.383702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57baa1-8f22-400c-af6b-d35b03199b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:09.385067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57baa1-8f22-400c-af6b-d35b03199b57 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:09.390357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 399/795] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:09.391042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a0479-24e3-4524-a16f-7d88175b451e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.391150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a0479-24e3-4524-a16f-7d88175b451e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.391408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a0479-24e3-4524-a16f-7d88175b451e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.393508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 400/796] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:09.398352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d097e-b4a6-4104-b47b-b9f2df2c8cbd None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.398940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d097e-b4a6-4104-b47b-b9f2df2c8cbd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.398940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d097e-b4a6-4104-b47b-b9f2df2c8cbd None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.401233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 401/797] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:09.401233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d097e-b4a6-4104-b47b-b9f2df2c8cbd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:09.401797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 397/798] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:09.403141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ddcc-22a2-4578-b411-393d0efdc7d2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.403441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ddcc-22a2-4578-b411-393d0efdc7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.403724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9ddcc-22a2-4578-b411-393d0efdc7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.425285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-32a9ddcc-22a2-4578-b411-393d0efdc7d2 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:09.426541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-32a9ddcc-22a2-4578-b411-393d0efdc7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:09.497136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:09.506237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 398/799] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:09.506911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9ddcc-22a2-4578-b411-393d0efdc7d2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:09.528552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d097e-b4a6-4104-b47b-b9f2df2c8cbd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.563516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:09.570875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ba96f-d84b-4aab-a0b8-cf2c7206de7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:09.571666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ba96f-d84b-4aab-a0b8-cf2c7206de7c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.571950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ba96f-d84b-4aab-a0b8-cf2c7206de7c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.572236 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ba96f-d84b-4aab-a0b8-cf2c7206de7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.579095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ba96f-d84b-4aab-a0b8-cf2c7206de7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:09.580548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ba96f-d84b-4aab-a0b8-cf2c7206de7c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-d93ba96f-d84b-4aab-a0b8-cf2c7206de7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:09.595090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:09.596205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:09.597459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 402/800] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:09.598082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fda66-3a95-4ac9-a907-879c8fe3d52f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.598082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fda66-3a95-4ac9-a907-879c8fe3d52f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.598374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34fda66-3a95-4ac9-a907-879c8fe3d52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.616022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f34fda66-3a95-4ac9-a907-879c8fe3d52f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:09.617266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f34fda66-3a95-4ac9-a907-879c8fe3d52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:09.617368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9ddcc-22a2-4578-b411-393d0efdc7d2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.655086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:09.655842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.656126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.656404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.663966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:09.665513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:09.680249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:09.681293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:09.682304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 399/801] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:10:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:09.682906 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:09.695848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 403/802] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:09.696899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fda66-3a95-4ac9-a907-879c8fe3d52f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:09.754263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:09.802326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137506dc-21ac-47b6-9f1e-b41828eb0d6f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.814680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34fda66-3a95-4ac9-a907-879c8fe3d52f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.836731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:09.839019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e2d3f-e8b6-480d-b387-24b980a7608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:09.839972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e2d3f-e8b6-480d-b387-24b980a7608b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.840289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e2d3f-e8b6-480d-b387-24b980a7608b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.840606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e2d3f-e8b6-480d-b387-24b980a7608b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.851415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e2d3f-e8b6-480d-b387-24b980a7608b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:09.851829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:09.852519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:09.852804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:09.853083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:09.853936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:09.854252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:09.855697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-797e2d3f-e8b6-480d-b387-24b980a7608b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:09.859176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:09.860113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:10:09.860984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:09.865651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 400/803] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:10:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:09.866531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e2d3f-e8b6-480d-b387-24b980a7608b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:09.872950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 404/804] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:09.872950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:09.977603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e2d3f-e8b6-480d-b387-24b980a7608b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.980565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082888-2581-4a38-b5f5-4abcec4dca82 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:09.994708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:10.014381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.015245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.015547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.015893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.017396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:10.017614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.017947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.018647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.019022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.019391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.020163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.020593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.021938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:10.035021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:10.036330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 17:10:10.037418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:10.047418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 401/805] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:10:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:10.047418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.047796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 405/806] 10.222.0.66 () {56 vars in 1129 bytes} [Thu Jun 16 17:10:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:10.047796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.096031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:10.150701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8d60-fa69-4b4e-bc54-f4db43d9b81e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:10.155212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25343c6c-f9ef-4a25-8202-85147839f44e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:10.189732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.190418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.190698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.190973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.191277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea837e-8ec5-48af-a1e0-37b341152a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.191728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:10.192031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.192501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea837e-8ec5-48af-a1e0-37b341152a87 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.192871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea837e-8ec5-48af-a1e0-37b341152a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.193285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea837e-8ec5-48af-a1e0-37b341152a87 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/0f15957f640a411eb4ac8861454f2b97/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.193677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.195495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea837e-8ec5-48af-a1e0-37b341152a87 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=0f15957f640a411eb4ac8861454f2b97, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.197039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea837e-8ec5-48af-a1e0-37b341152a87 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.205217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 402/807] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:10:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:10.205217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.225232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 406/808] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:10:10 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/0f15957f640a411eb4ac8861454f2b97/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:10:10.225232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea837e-8ec5-48af-a1e0-37b341152a87 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.251597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:10.336473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95591aff-630d-4e53-b662-b036ee34b39d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:10.359950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea837e-8ec5-48af-a1e0-37b341152a87 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:10.363328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:10.378751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.379475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.380072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.380072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.380891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:10.381299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.382783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.393211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 403/809] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Jun 16 17:10:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:10.393211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.410255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d8244f-6e10-4644-a302-e913678faf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.410977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8244f-6e10-4644-a302-e913678faf0f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.411368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8244f-6e10-4644-a302-e913678faf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.411675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d8244f-6e10-4644-a302-e913678faf0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.483636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8244f-6e10-4644-a302-e913678faf0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.485100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d8244f-6e10-4644-a302-e913678faf0f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.528078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:10.531662 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4545a1-8225-4349-8d92-6cd1c962204f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:10.567422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.568497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.568779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.569054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.595398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 407/810] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:10.597368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6482d-5487-41f8-a51d-b82a8ffecd82 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.597875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6482d-5487-41f8-a51d-b82a8ffecd82 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.598351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d6482d-5487-41f8-a51d-b82a8ffecd82 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.604175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 408/811] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:10.604175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6482d-5487-41f8-a51d-b82a8ffecd82 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.609328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:10.704134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.705684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.720773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d6482d-5487-41f8-a51d-b82a8ffecd82 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:10.724082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:10.725140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:10.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:10.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:10.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:10.726633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:10.730237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 404/812] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:10.730856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.762073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b4114f-d1b3-4603-ad38-422b754c47fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.762767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4114f-d1b3-4603-ad38-422b754c47fb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.763053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4114f-d1b3-4603-ad38-422b754c47fb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.763332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b4114f-d1b3-4603-ad38-422b754c47fb None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.764049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4114f-d1b3-4603-ad38-422b754c47fb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:10.764434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4114f-d1b3-4603-ad38-422b754c47fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.765850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b4114f-d1b3-4603-ad38-422b754c47fb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.772640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 409/813] 10.222.0.66 () {56 vars in 1124 bytes} [Thu Jun 16 17:10:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:10.775148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0fd2-cf51-4ba9-82e7-1207c3c3edcb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.776065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0fd2-cf51-4ba9-82e7-1207c3c3edcb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.777146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d0fd2-cf51-4ba9-82e7-1207c3c3edcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.796096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8e1d0fd2-cf51-4ba9-82e7-1207c3c3edcb None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:10.797277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8e1d0fd2-cf51-4ba9-82e7-1207c3c3edcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:10.837721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8e8f8-3af2-4dca-8d61-255b11efbb54 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:10.861746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:10.869042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 410/814] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:10.873566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:10.874371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:10.874648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:10.874916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:10.881151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d0fd2-cf51-4ba9-82e7-1207c3c3edcb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.885697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:10.887151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:10.905590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:10.906598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:10.907520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:10.907520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:10.907520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:10.907520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:10.911524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 405/815] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:10.912216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:10.917018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:10.995254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d0fd2-cf51-4ba9-82e7-1207c3c3edcb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.019178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f04c9-cc15-4557-9c98-48cd03530c62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.031674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.032380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.032662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.032932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.033986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:11.033986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.035391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.047468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 411/816] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:11.048625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.062671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e464833-d09c-4ce2-a233-830e7a2dca9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.063503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464833-d09c-4ce2-a233-830e7a2dca9c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.063781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464833-d09c-4ce2-a233-830e7a2dca9c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.064050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e464833-d09c-4ce2-a233-830e7a2dca9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.064870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e464833-d09c-4ce2-a233-830e7a2dca9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.066344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e464833-d09c-4ce2-a233-830e7a2dca9c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.097205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 406/817] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:10:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:11.097205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e464833-d09c-4ce2-a233-830e7a2dca9c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.131097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:11.170688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:11.197519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b90d41-8427-48f9-958b-8c20b0d0636d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.228153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e464833-d09c-4ce2-a233-830e7a2dca9c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.248559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.249838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.250317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.250785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] PATH_INFO: `/v3/services/network` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.252256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.254685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.266911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.267607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.267889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.268165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.268931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:11.269262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.270733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:11.272480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:11.273506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 17:10:11.274392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:11.282426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 412/818] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:11.282426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.288094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 407/819] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:11.288779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.412419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd6512-c90e-4efc-9930-e07e3d4337fb None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.423844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16b74b-ca6f-44e8-bf86-9b6e9101fa6a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.437434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:11.445580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:11.451585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.452765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.453277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.453744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.463784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.466087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.466284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.466907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.467194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.467462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.468225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] RBAC: Adding query filter params (name=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:11.468531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.473582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.483879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 408/820] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:11.483879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:11.493382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:11.494338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:11.495297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:11.498160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 413/821] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:11.499247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.611486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9ba54-2b60-4c20-b47e-9c5a3768a7bf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.618500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e39ae-06bb-4731-aa23-d700ece12023 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.649478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.650693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.651173 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.651664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.659893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.660022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.660686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.661207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.661545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.662295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] RBAC: Adding query filter params (type=network) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:11.662614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.662719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.663969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.670779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 414/822] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:11.671653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:11.678653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:11.679580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:11.680513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:11.680513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:11.680513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:11.680513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:11.680513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:11.680513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:11.680513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:11.680513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:11.683045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 409/823] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:11.684007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.737156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:11.753698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:11.794841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18b803-4bd2-4b23-9d3e-d32683a92a48 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.809360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd0442-79a9-4d52-83ac-91773f515255 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:11.831166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a987b-f0fb-48ea-83a8-94e1d6e436b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.832224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a987b-f0fb-48ea-83a8-94e1d6e436b7 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.832739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a987b-f0fb-48ea-83a8-94e1d6e436b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.833324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a987b-f0fb-48ea-83a8-94e1d6e436b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.834279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a987b-f0fb-48ea-83a8-94e1d6e436b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.836317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a987b-f0fb-48ea-83a8-94e1d6e436b7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.854297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:11.855021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:11.855307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:11.855581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:11.856547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:11.856861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:11.858498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:11.859139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 410/824] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:10:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:11.860186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a987b-f0fb-48ea-83a8-94e1d6e436b7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.882881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 415/825] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:11.888765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:11.990202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:12.010339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a987b-f0fb-48ea-83a8-94e1d6e436b7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.022652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bbdae-6094-4009-96c7-6aaab5b7745e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.025645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:12.059233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.060298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.060816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.061335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.062335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:12.062846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:12.064500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:12.072116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.072953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.073262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.073536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.080319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 411/826] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:12.093399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:12.094681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:12.094920 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:12.117434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:12.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:12.119358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:12.119358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:12.119358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:12.119358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:12.119358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 416/827] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:12.127571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:12.232539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547b979-4428-4e77-be13-4bc618ff1093 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.253037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb4b6a4-0c8e-49c9-ae5e-ddcd1bed0a6a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.277908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.278584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.278869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.279152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.279991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=e3241452a88a40da9c9b8fa79c4fc9c0) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:12.280305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:12.283229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:12.292417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 412/828] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:12 2022] GET /identity/v3/role_assignments?user.id=e3241452a88a40da9c9b8fa79c4fc9c0&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:12.302301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51183cc-e82b-4fe7-b38d-a92e41607572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.303321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51183cc-e82b-4fe7-b38d-a92e41607572 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.303759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51183cc-e82b-4fe7-b38d-a92e41607572 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.304182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51183cc-e82b-4fe7-b38d-a92e41607572 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.305117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51183cc-e82b-4fe7-b38d-a92e41607572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:12.305596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51183cc-e82b-4fe7-b38d-a92e41607572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:12.307156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51183cc-e82b-4fe7-b38d-a92e41607572 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:12.322089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 417/829] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:12.327602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:12.348594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:12.444779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad6e8d-3652-4dea-9f28-ddd9ab58551d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.474743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a114aa-9c0f-4b9c-806d-b8f9c6312cca None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.475317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a114aa-9c0f-4b9c-806d-b8f9c6312cca None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.475317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a114aa-9c0f-4b9c-806d-b8f9c6312cca None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.477003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 418/830] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:12.482608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80e71e-3f57-406c-90e6-005f2a336413 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.482901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80e71e-3f57-406c-90e6-005f2a336413 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.483183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80e71e-3f57-406c-90e6-005f2a336413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.489977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.491197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.491672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.492130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.501798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:12.503633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:12.504518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3c80e71e-3f57-406c-90e6-005f2a336413 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:12.506158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3c80e71e-3f57-406c-90e6-005f2a336413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:12.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:12.520926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:12.522140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:12.522140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:12.522140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:12.522140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:12.522140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:12.522140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:12.522140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:12.522140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:12.524552 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 413/831] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:12.524720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:12.529639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:12.606854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 419/832] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:12.627284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:12.630690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80e71e-3f57-406c-90e6-005f2a336413 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:12.659075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80959150-7154-4577-b035-4efb4806edb0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.697293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.698482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.698983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.699528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.700646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:12.701189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:12.702884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:12.717454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 414/833] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:12.728300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:12.752072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80e71e-3f57-406c-90e6-005f2a336413 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.796156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d22d-126c-473a-858d-bd82cb904000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.797385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d22d-126c-473a-858d-bd82cb904000 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.797787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d22d-126c-473a-858d-bd82cb904000 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.798322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d22d-126c-473a-858d-bd82cb904000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.813899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6d22d-126c-473a-858d-bd82cb904000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-36b6d22d-126c-473a-858d-bd82cb904000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:12.823731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:12.824714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:12.825677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:12.826348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 420/834] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:12.832206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d22d-126c-473a-858d-bd82cb904000 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:12.844024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:12.855386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c28f8a-bc35-4a40-afc7-b290b7ae6d38 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.893442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc290aa5-b5d0-432c-bc33-96672e45e1b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.894709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc290aa5-b5d0-432c-bc33-96672e45e1b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.895342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc290aa5-b5d0-432c-bc33-96672e45e1b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.895816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc290aa5-b5d0-432c-bc33-96672e45e1b4 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/ae4e4bbf4359400289b9fd2ba8778067/roles/3b84c0487e23407e8452c17b5886b640` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.899514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:12.915347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc290aa5-b5d0-432c-bc33-96672e45e1b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=ae4e4bbf4359400289b9fd2ba8778067, role_id=3b84c0487e23407e8452c17b5886b640)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:12.916912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc290aa5-b5d0-432c-bc33-96672e45e1b4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:12.935262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 415/835] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:10:12 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/ae4e4bbf4359400289b9fd2ba8778067/roles/3b84c0487e23407e8452c17b5886b640 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:10:12.953725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d22d-126c-473a-858d-bd82cb904000 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:12.994743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:12.995435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:12.995717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:12.995993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:12.996783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:12.997089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:12.998626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:13.009089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 421/836] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:13.014522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc290aa5-b5d0-432c-bc33-96672e45e1b4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:13.135457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc290aa5-b5d0-432c-bc33-96672e45e1b4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:13.171374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:13.172230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:13.172508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:13.172777 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:13.184088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:13.185560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:13.204678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:13.205692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:13.206612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:13.206612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:13.206612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:13.206612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:13.206612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 416/837] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:13.212154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:13.283474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:13.322875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295339d-df7b-44b9-b9d5-1ad448df0dc2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:13.358631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:13.359301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:13.359582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:13.359855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:13.360576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:13.360881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:13.362282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:13.375995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 422/838] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:13.381697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:13.423257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:13.493828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d555d1-8b42-4339-929c-0bf752b6e91d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:13.533465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b8514-d71b-46b2-95a0-9345d98b76b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:13.534295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8514-d71b-46b2-95a0-9345d98b76b1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:13.534577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8514-d71b-46b2-95a0-9345d98b76b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:13.534913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b8514-d71b-46b2-95a0-9345d98b76b1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:13.543715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b8514-d71b-46b2-95a0-9345d98b76b1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:13.545190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b8514-d71b-46b2-95a0-9345d98b76b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6d1b8514-d71b-46b2-95a0-9345d98b76b1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:13.561420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:13.562457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:13.563416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 417/839] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:13.568800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:13.583235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c930a5-5702-4742-99b6-6f37da5a3ea5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:13.583521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c930a5-5702-4742-99b6-6f37da5a3ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:13.583784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c930a5-5702-4742-99b6-6f37da5a3ea5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:13.585880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 418/840] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:13.590708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92e8b9-c995-4050-9ebd-9261647ebae6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:13.590995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92e8b9-c995-4050-9ebd-9261647ebae6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:13.591271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f92e8b9-c995-4050-9ebd-9261647ebae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:13.609449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1f92e8b9-c995-4050-9ebd-9261647ebae6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:13.610676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f92e8b9-c995-4050-9ebd-9261647ebae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:13.693520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 419/841] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:13.708785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92e8b9-c995-4050-9ebd-9261647ebae6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:13.744354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:13.824767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f92e8b9-c995-4050-9ebd-9261647ebae6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:13.843672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5644d301-8667-4625-b42c-1644fd72c1e7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:13.861323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:13.862460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:13.862944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:13.863553 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:13.864813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:13.866414 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:13.874140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:13.881046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:13.882296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:13.883184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:13.884404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:13.885139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:13.885511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:13.885849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:13.886648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 420/842] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:13.889647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:13.890129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:13.891822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:13.892043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:13.915224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 423/843] 10.222.0.66 () {56 vars in 1135 bytes} [Thu Jun 16 17:10:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:13.916474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:13.993901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:14.005079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3981424-db5e-4fcc-a339-657d15f4219c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:14.017407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf0089-fc6c-4d2f-b2d3-c676f23d8d8c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:14.043171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:14.043956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.044241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.044516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.045258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:14.045598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:14.047252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:14.057435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 421/844] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:14.059162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:14.060174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db8c56-59d7-44e2-a015-b2fe2d5918a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:14.060850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db8c56-59d7-44e2-a015-b2fe2d5918a5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.061129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db8c56-59d7-44e2-a015-b2fe2d5918a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.061839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8db8c56-59d7-44e2-a015-b2fe2d5918a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.120332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:14.135256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db8c56-59d7-44e2-a015-b2fe2d5918a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:14.136710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8db8c56-59d7-44e2-a015-b2fe2d5918a5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:14.176751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ac0a1-fcca-49a2-ae59-845d28a7d972 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:14.216051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:14.217161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.217690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.218201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.219297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:14.219800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:14.221439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:14.232248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 422/845] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:14 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:14.240480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 424/846] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:14.246323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:14.281513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:14.363363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7107c61d-ec35-49c4-b94d-736b308ae4fe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:14.409966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:14.411008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.411329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.411624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.426146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:14.428582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:14.448972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:14.449976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:14.450887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:14.450887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:14.450887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:14.450887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:14.450887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 423/847] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:14.456320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db8c56-59d7-44e2-a015-b2fe2d5918a5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:14.557976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:14.575383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8db8c56-59d7-44e2-a015-b2fe2d5918a5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:14.611454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6bf82-581c-40c6-8e0e-acd822b9e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:14.612533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6bf82-581c-40c6-8e0e-acd822b9e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.613323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6bf82-581c-40c6-8e0e-acd822b9e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.613796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6bf82-581c-40c6-8e0e-acd822b9e6d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.614779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6bf82-581c-40c6-8e0e-acd822b9e6d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:14.615277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6bf82-581c-40c6-8e0e-acd822b9e6d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:14.616874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f6bf82-581c-40c6-8e0e-acd822b9e6d3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:14.630097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 425/848] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:14.636345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:14.718796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:14.722904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac350-911f-4d98-b618-4f25ab360b13 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.723741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac350-911f-4d98-b618-4f25ab360b13 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.724219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabac350-911f-4d98-b618-4f25ab360b13 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.726204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 426/849] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:14.732715 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f94f1-1fcb-4379-8291-94c578935574 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.732882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f94f1-1fcb-4379-8291-94c578935574 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.733068 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f94f1-1fcb-4379-8291-94c578935574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.751088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2a2f94f1-1fcb-4379-8291-94c578935574 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:14.752324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2a2f94f1-1fcb-4379-8291-94c578935574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:14.757470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee683e6a-54b4-4f25-bb8b-a4b25e672b22 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:14.798111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:14.798920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.799198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.799469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.807269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:14.809050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:14.826395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 427/850] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:14 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:14.831258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:14.832188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:14.833839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:14.833839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:14.833839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:14.833839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:14.833839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:14.833839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:14.833839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:14.833839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:14.836360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 424/851] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:14.842395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f94f1-1fcb-4379-8291-94c578935574 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:14.851269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:14.924400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:14.950427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f94f1-1fcb-4379-8291-94c578935574 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:14.979237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b24ce-207b-4af9-81b3-02f9108a8427 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:14.987190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:14.988211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:14.988685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:14.989294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:14.990345 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:14.990858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:14.992489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:15.009514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 428/852] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:15.011374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:15.018748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:15.019578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.019860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.020133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.021183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:15.022804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:15.025547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:15.038910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:15.040158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:15.041131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:15.043885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 425/853] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Jun 16 17:10:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:15.050495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:15.137737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec846ed3-8722-40cc-9ce0-3104e925cc18 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:15.174271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1965d-1cb5-41cd-8ecc-3071dcc95ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:15.174961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1965d-1cb5-41cd-8ecc-3071dcc95ead None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.175241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1965d-1cb5-41cd-8ecc-3071dcc95ead None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.175517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1965d-1cb5-41cd-8ecc-3071dcc95ead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.176809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1965d-1cb5-41cd-8ecc-3071dcc95ead None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=0f15957f640a411eb4ac8861454f2b97) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:15.177391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1965d-1cb5-41cd-8ecc-3071dcc95ead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:15.178727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1965d-1cb5-41cd-8ecc-3071dcc95ead None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:15.181616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:15.194270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 429/854] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:15 2022] GET /identity/v3/role_assignments?user.id=0f15957f640a411eb4ac8861454f2b97&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:15.195886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ed05-8cda-46e5-b0ad-db42c96b64e0 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.196181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ed05-8cda-46e5-b0ad-db42c96b64e0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.196462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ed05-8cda-46e5-b0ad-db42c96b64e0 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.196904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc1a71-7a66-401f-957b-34403e9db0ad None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:15.198777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 430/855] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:15.203646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb45f7-c76e-4b2e-b2ed-12ca80daf73b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.203934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb45f7-c76e-4b2e-b2ed-12ca80daf73b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.204218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb45f7-c76e-4b2e-b2ed-12ca80daf73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.229578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-88cb45f7-c76e-4b2e-b2ed-12ca80daf73b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:15.230984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-88cb45f7-c76e-4b2e-b2ed-12ca80daf73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:15.245461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:15.246381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.247086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.247372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.248376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:15.248696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:15.250715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:15.261358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 426/856] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:15.265061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:15.321625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 431/857] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:15.338866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb45f7-c76e-4b2e-b2ed-12ca80daf73b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:15.347925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:15.406310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd742276-342c-4915-81c7-9a397bd0daf5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:15.455960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:15.457023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.461412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.461999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.462812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb45f7-c76e-4b2e-b2ed-12ca80daf73b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:15.463216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:15.463801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:15.466754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:15.475779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 427/858] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:15 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:15.503429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:15.510714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d7223-9541-41de-ba3e-fcfb80c2f382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:15.511574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7223-9541-41de-ba3e-fcfb80c2f382 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.511854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7223-9541-41de-ba3e-fcfb80c2f382 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.512126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7223-9541-41de-ba3e-fcfb80c2f382 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.525408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d7223-9541-41de-ba3e-fcfb80c2f382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-116d7223-9541-41de-ba3e-fcfb80c2f382 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:15.533507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:15.534541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:15.535422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 432/859] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:10:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:15.541431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:15.654741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b9f58-8238-4040-b7b5-7b9958a06d4f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:15.681096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:15.695371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:15.696061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.696342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.696617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.697419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:15.697721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:15.699114 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:15.708503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 428/860] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:10:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:15.714441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d7223-9541-41de-ba3e-fcfb80c2f382 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:15.827316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d7223-9541-41de-ba3e-fcfb80c2f382 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:15.862677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:15.863483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:15.863767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:15.864119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:15.875460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:15.876936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:15.894383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:15.895407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:15.896354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:15.896354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:15.896354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:15.896354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:15.896354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 433/861] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:15.901670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:16.011417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc5c8c2-2dc1-4ebe-93a3-1eb294f2722a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:16.048002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cd991-f047-4218-8567-227169670390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:16.049050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cd991-f047-4218-8567-227169670390 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:16.049554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cd991-f047-4218-8567-227169670390 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:16.050036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cd991-f047-4218-8567-227169670390 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:16.051073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cd991-f047-4218-8567-227169670390 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:16.051639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cd991-f047-4218-8567-227169670390 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:16.053317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cd991-f047-4218-8567-227169670390 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:16.067945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 429/862] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:16.074090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:16.121113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:16.179223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976875a1-a8aa-4b16-aba5-7030d3b094db None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:16.215101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:16.215971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:16.216253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:16.216525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:16.224727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:16.226313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:16.241524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:16.242521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:16.243493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 434/863] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:10:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:16.249235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cd991-f047-4218-8567-227169670390 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:16.299334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:16.360051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cd991-f047-4218-8567-227169670390 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:16.398762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5b784-d91a-4226-b3d6-ddd14976425b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:16.399468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5b784-d91a-4226-b3d6-ddd14976425b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:16.399750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5b784-d91a-4226-b3d6-ddd14976425b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:16.400021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5b784-d91a-4226-b3d6-ddd14976425b None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:16.400884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5b784-d91a-4226-b3d6-ddd14976425b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:16.401204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5b784-d91a-4226-b3d6-ddd14976425b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:16.402646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a5b784-d91a-4226-b3d6-ddd14976425b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:16.421760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 430/864] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Jun 16 17:10:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:16.427471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:16.520572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f79822-8800-4dc5-ba9b-a9c60c08d39e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:16.520938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f79822-8800-4dc5-ba9b-a9c60c08d39e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:16.521213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f79822-8800-4dc5-ba9b-a9c60c08d39e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:16.522759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 431/865] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:16.528514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018966-8414-42ab-a31b-042809e7766d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:16.529061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018966-8414-42ab-a31b-042809e7766d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:16.529661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91018966-8414-42ab-a31b-042809e7766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:16.545167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195ae1e-5fc4-492c-b4b2-20f0c2ad3774 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:16.546783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:16.548174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-91018966-8414-42ab-a31b-042809e7766d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:16.550042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-91018966-8414-42ab-a31b-042809e7766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:16.580145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0ed41-09b3-4551-b9d4-5095ed436448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:16.580802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0ed41-09b3-4551-b9d4-5095ed436448 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:16.581087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0ed41-09b3-4551-b9d4-5095ed436448 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:16.581395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa0ed41-09b3-4551-b9d4-5095ed436448 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:16.630661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 432/866] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:16.650087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91018966-8414-42ab-a31b-042809e7766d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:16.657384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0ed41-09b3-4551-b9d4-5095ed436448 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:16.658870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa0ed41-09b3-4551-b9d4-5095ed436448 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:16.686748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:16.763541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91018966-8414-42ab-a31b-042809e7766d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:16.784204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 435/867] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:16.789729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0ed41-09b3-4551-b9d4-5095ed436448 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:16.791798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:16.807779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde23ca-50ef-4faf-aa55-657a34b5734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:16.808703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde23ca-50ef-4faf-aa55-657a34b5734d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:16.809195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde23ca-50ef-4faf-aa55-657a34b5734d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:16.809650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde23ca-50ef-4faf-aa55-657a34b5734d None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:16.810701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde23ca-50ef-4faf-aa55-657a34b5734d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:16.812389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde23ca-50ef-4faf-aa55-657a34b5734d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:16.837574 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 433/868] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:10:16 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:16.906280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa0ed41-09b3-4551-b9d4-5095ed436448 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:16.952012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:16.952992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:16.953389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:16.953740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:16.966458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:16.968141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:16.987865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:16.988873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:16.989996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:16.989996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:16.989996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:16.989996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:16.989996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 436/869] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:16.995470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde23ca-50ef-4faf-aa55-657a34b5734d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:17.094175 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:17.103543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde23ca-50ef-4faf-aa55-657a34b5734d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:17.138677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:17.139376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.139658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.139933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.140663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:17.140971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:17.142636 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:17.155688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 434/870] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:17.161351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:17.214575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:17.266615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f72d95-09bc-4171-91e4-7d85cf13b54c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:17.303065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf7a42-d717-4900-97c8-db9434745238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:17.303926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7a42-d717-4900-97c8-db9434745238 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.304209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7a42-d717-4900-97c8-db9434745238 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.304484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf7a42-d717-4900-97c8-db9434745238 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.311688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf7a42-d717-4900-97c8-db9434745238 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:17.313138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf7a42-d717-4900-97c8-db9434745238 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-daaf7a42-d717-4900-97c8-db9434745238 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:17.327300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:17.328248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:17.329154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 437/871] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:17.335159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:17.446812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57795446-8390-4153-8809-0c53479c25e6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:17.457695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0caf5-49bb-4c22-a81b-47c07ba919a3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.458048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0caf5-49bb-4c22-a81b-47c07ba919a3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.458331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0caf5-49bb-4c22-a81b-47c07ba919a3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.460747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 438/872] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:17.466935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2326-cd5a-484c-b24e-df064b4f27b0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.467221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2326-cd5a-484c-b24e-df064b4f27b0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.467500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2326-cd5a-484c-b24e-df064b4f27b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.497863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:17.498256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-fe2f2326-cd5a-484c-b24e-df064b4f27b0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:17.498748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.499125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.499516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.500103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fe2f2326-cd5a-484c-b24e-df064b4f27b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:17.500528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:17.500938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:17.502527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:17.521285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 435/873] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:17.524322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:17.552002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:17.609320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 439/874] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:17.629766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f2326-cd5a-484c-b24e-df064b4f27b0 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:17.665065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7caf72b-b41d-4b1b-88f4-7a9382ba920f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:17.683993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:17.705803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5138d0-030f-4a77-98f1-efde3e76bc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:17.706923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5138d0-030f-4a77-98f1-efde3e76bc3b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.707416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5138d0-030f-4a77-98f1-efde3e76bc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.708029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5138d0-030f-4a77-98f1-efde3e76bc3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.709288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5138d0-030f-4a77-98f1-efde3e76bc3b None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=ae4e4bbf4359400289b9fd2ba8778067) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:17.710161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5138d0-030f-4a77-98f1-efde3e76bc3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:17.711886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5138d0-030f-4a77-98f1-efde3e76bc3b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:17.725331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 436/875] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:17 2022] GET /identity/v3/role_assignments?user.id=ae4e4bbf4359400289b9fd2ba8778067&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:17.761756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f2326-cd5a-484c-b24e-df064b4f27b0 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:17.786477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ee8d1-f7a6-4141-8434-b1e29a32af7b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.786986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ee8d1-f7a6-4141-8434-b1e29a32af7b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.787386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ee8d1-f7a6-4141-8434-b1e29a32af7b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.793129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 437/876] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:17.794968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7aebe-257d-4e5e-b128-1d3633bbcc90 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.795257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7aebe-257d-4e5e-b128-1d3633bbcc90 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.795538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe7aebe-257d-4e5e-b128-1d3633bbcc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.802993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:17.803845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.804129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.804403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.805066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:17.806506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:17.813911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ffe7aebe-257d-4e5e-b128-1d3633bbcc90 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:17.815143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ffe7aebe-257d-4e5e-b128-1d3633bbcc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:17.820192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:17.821356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:17.822286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 440/877] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:17.827847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:17.909608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 438/878] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:17 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:17.933118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7aebe-257d-4e5e-b128-1d3633bbcc90 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:17.943729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:17.952204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea7402c-9cc3-44d7-848b-e993b5c12dd6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:17.956051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:17.997429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:17.998129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:17.998414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:17.998691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:17.999435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:17.999756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:18.001129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:18.007837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 441/879] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:18.013088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:18.060524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe7aebe-257d-4e5e-b128-1d3633bbcc90 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:18.099627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862ef18-cf55-4735-983c-e99ef2f92d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:18.100327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862ef18-cf55-4735-983c-e99ef2f92d77 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:18.100612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862ef18-cf55-4735-983c-e99ef2f92d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:18.100891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862ef18-cf55-4735-983c-e99ef2f92d77 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:18.101658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862ef18-cf55-4735-983c-e99ef2f92d77 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:18.103096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862ef18-cf55-4735-983c-e99ef2f92d77 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:18.125275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 439/880] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:10:17 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:18.153033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7004d7-7981-4255-a6fd-e4338485bc85 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:18.199384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bebf479-815d-4f51-b36b-741c1ad087b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:18.200510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebf479-815d-4f51-b36b-741c1ad087b1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:18.201123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebf479-815d-4f51-b36b-741c1ad087b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:18.201642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bebf479-815d-4f51-b36b-741c1ad087b1 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:18.202644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebf479-815d-4f51-b36b-741c1ad087b1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:18.203181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebf479-815d-4f51-b36b-741c1ad087b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:18.204850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bebf479-815d-4f51-b36b-741c1ad087b1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:18.211612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 442/881] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:19.087651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8b009-7164-48b8-82b7-9e962b20f32f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:19.088229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8b009-7164-48b8-82b7-9e962b20f32f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:19.088229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c8b009-7164-48b8-82b7-9e962b20f32f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:19.089834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 440/882] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:19.095184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea745-a81e-4e9d-af1a-785848c261dd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:19.095595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea745-a81e-4e9d-af1a-785848c261dd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:19.095775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cea745-a81e-4e9d-af1a-785848c261dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:19.112605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c9cea745-a81e-4e9d-af1a-785848c261dd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:19.113846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c9cea745-a81e-4e9d-af1a-785848c261dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:19.138217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:19.193268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 443/883] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:19.215836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8b009-7164-48b8-82b7-9e962b20f32f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:19.288217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:19.325157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c8b009-7164-48b8-82b7-9e962b20f32f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:19.371431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:19.372222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:19.372509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:19.372788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:19.373499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:19.374886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:19.390637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:19.391754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:19.392661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 441/884] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:19.398322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cea745-a81e-4e9d-af1a-785848c261dd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:19.505296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cea745-a81e-4e9d-af1a-785848c261dd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:19.541733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:19.542415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:19.542698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:19.542973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:19.543700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:19.544009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:19.545410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:19.550867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 444/885] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:19.556387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:19.673271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a17b1a-80e0-4251-83b6-82b86da9e76b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:19.705090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:19.712061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:19.712920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:19.713365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:19.713808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:19.714690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:19.715152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:19.716663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:19.722757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 442/886] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:19 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:19.728800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:19.754970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:19.844005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443e313-ba35-4b76-9f9c-3271223efa42 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:19.885418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd124ec-f1b2-43f2-906d-143d74666fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:19.886090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd124ec-f1b2-43f2-906d-143d74666fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:19.886376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd124ec-f1b2-43f2-906d-143d74666fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:19.886656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd124ec-f1b2-43f2-906d-143d74666fd4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:19.887465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd124ec-f1b2-43f2-906d-143d74666fd4 None admin] RBAC: Adding query filter params (interface=public, service_id=6d298244b874423f84042bd7cbe04fc3, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:19.887777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd124ec-f1b2-43f2-906d-143d74666fd4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:19.889170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd124ec-f1b2-43f2-906d-143d74666fd4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:19.895280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 445/887] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Jun 16 17:10:19 2022] GET /identity/v3/endpoints?service_id=6d298244b874423f84042bd7cbe04fc3&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:19.903270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:20.023795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b664cf-66e2-43a5-bba9-4b954402587e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:20.062675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c5dbe-f2f0-49e1-862b-c75a9a46a6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:20.063728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c5dbe-f2f0-49e1-862b-c75a9a46a6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.064204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c5dbe-f2f0-49e1-862b-c75a9a46a6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.064639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c5dbe-f2f0-49e1-862b-c75a9a46a6e9 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.065517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c5dbe-f2f0-49e1-862b-c75a9a46a6e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:20.067228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c5dbe-f2f0-49e1-862b-c75a9a46a6e9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:20.076775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 443/888] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:10:19 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:10:20.190206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508df1a-85b5-480e-97da-0db17c16e051 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.190809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508df1a-85b5-480e-97da-0db17c16e051 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.190809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508df1a-85b5-480e-97da-0db17c16e051 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.192834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 446/889] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:20.198716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ac7f-d5b3-4b7d-bd94-5c038e98ffff None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.199274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ac7f-d5b3-4b7d-bd94-5c038e98ffff None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.199274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a4ac7f-d5b3-4b7d-bd94-5c038e98ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.219309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-60a4ac7f-d5b3-4b7d-bd94-5c038e98ffff None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:20.220565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-60a4ac7f-d5b3-4b7d-bd94-5c038e98ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:20.286314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:20.299816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 444/890] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:20.325324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508df1a-85b5-480e-97da-0db17c16e051 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:20.442172 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:20.453986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac4c6b-0009-46e1-b1b1-040d87e0ccf7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.454287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac4c6b-0009-46e1-b1b1-040d87e0ccf7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.454697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac4c6b-0009-46e1-b1b1-040d87e0ccf7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.456353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 445/891] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:20.462446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18267a-8d6a-401a-ad2b-dc5e7d63bc0c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.462783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18267a-8d6a-401a-ad2b-dc5e7d63bc0c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.462998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18267a-8d6a-401a-ad2b-dc5e7d63bc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.491692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6f18267a-8d6a-401a-ad2b-dc5e7d63bc0c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:20.493489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f18267a-8d6a-401a-ad2b-dc5e7d63bc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:20.494108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508df1a-85b5-480e-97da-0db17c16e051 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:20.551174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbec64e-3d5e-4a07-b5f4-0fc99fd1241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:20.553070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec64e-3d5e-4a07-b5f4-0fc99fd1241a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.553390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec64e-3d5e-4a07-b5f4-0fc99fd1241a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.553986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbec64e-3d5e-4a07-b5f4-0fc99fd1241a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.568343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbec64e-3d5e-4a07-b5f4-0fc99fd1241a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:20.571724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 446/892] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:20.574702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09225-d5dd-4983-996e-d3bb41565cc7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.575219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09225-d5dd-4983-996e-d3bb41565cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.575709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b09225-d5dd-4983-996e-d3bb41565cc7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-cbbec64e-3d5e-4a07-b5f4-0fc99fd1241a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:20.576994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:20.577962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:20.579391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 447/893] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Jun 16 17:10:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:20.580708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 447/894] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:20 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:20.584141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbec64e-3d5e-4a07-b5f4-0fc99fd1241a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:20.588724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3db554-dfd2-4d25-925a-bf4f366c21ea None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.589660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3db554-dfd2-4d25-925a-bf4f366c21ea None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.590527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3db554-dfd2-4d25-925a-bf4f366c21ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.609654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fd3db554-dfd2-4d25-925a-bf4f366c21ea None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:20.610840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd3db554-dfd2-4d25-925a-bf4f366c21ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:20.684295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 448/895] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:20.685510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3db554-dfd2-4d25-925a-bf4f366c21ea None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:20.720521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbec64e-3d5e-4a07-b5f4-0fc99fd1241a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:20.745248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:20.760806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:20.761643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.761924 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.762316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.763093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:20.763415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:20.764792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:20.779535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 448/896] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Jun 16 17:10:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:20.779535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:20.789476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:20.807650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3db554-dfd2-4d25-925a-bf4f366c21ea None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:20.842913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:20.843755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.844033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.844304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.844964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:20.846369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:20.862018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:20.863169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:20.864312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:20.866561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 449/897] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Jun 16 17:10:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:20.867607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:20.892274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fdcd89-9a6c-4fa1-9329-3401707ce0d2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:20.929198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a7844-4670-4e6a-8cb0-f7a9fa725230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:20.929896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a7844-4670-4e6a-8cb0-f7a9fa725230 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:20.930182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a7844-4670-4e6a-8cb0-f7a9fa725230 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:20.930458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a7844-4670-4e6a-8cb0-f7a9fa725230 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:20.931195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a7844-4670-4e6a-8cb0-f7a9fa725230 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:20.932605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a7844-4670-4e6a-8cb0-f7a9fa725230 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:20.957219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 449/898] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:10:20 2022] POST /identity/v3/services => generated 241 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:20.957219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a7844-4670-4e6a-8cb0-f7a9fa725230 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:20.979764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade4d31-dc7c-40ee-9a1c-a160c73e1fce None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.000002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:21.035888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.037287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.037642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.038007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.047868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:21.053927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.055477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:21.076645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:21.077704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:21.078633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:21.078633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:21.078633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:21.078633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:21.083356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 450/899] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:21.084408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:21.088373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a7844-4670-4e6a-8cb0-f7a9fa725230 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.127109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.127870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.128142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.128417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.129261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:21.129653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.132015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:21.139114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 450/900] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:21.144317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:21.207377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76e790-eeb0-4730-9770-4ea6d73f32f2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.243494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.244163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.244441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.244711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.245491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:21.245802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.247193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:21.259680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 451/901] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:21.265778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:21.274335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b17d487-dc18-4fec-b479-04c2927a841a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.295820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:21.308663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:21.314861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.315633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.315977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.316320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.317547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:21.317918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.319380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:21.325503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 451/902] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:21.331349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:21.390967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c723a77-0c20-4464-a6b8-2ec1c9fe41dc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.426885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.427739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.428015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.428364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.435941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.437268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:21.453751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:21.454776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:21.456339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 452/903] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:10:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:21.467894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775adce-31a6-4a09-9b8f-c550c77be811 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.470350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:21.530713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.531416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.531699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.531975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.533069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] RBAC: Adding query filter params (interface=public, service_id=f499518b24c64be6bccad99501fb799a, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:21.533436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.535138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:21.545209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 452/904] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Jun 16 17:10:21 2022] GET /identity/v3/endpoints?service_id=f499518b24c64be6bccad99501fb799a&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:21.548631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:21.579781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:21.606858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fae45c-516e-417a-af49-fd9178c564e1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.618395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:21.655281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.656802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.657317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.658005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.659248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:21.660121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.662326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:21.662849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9095-432c-47f3-8ee8-81717724f555 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.690849 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 453/905] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Jun 16 17:10:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:21.692243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:21.711553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b8a7a-34a9-4be6-b599-99b7de12645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.712820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b8a7a-34a9-4be6-b599-99b7de12645a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.713569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b8a7a-34a9-4be6-b599-99b7de12645a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.714045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b8a7a-34a9-4be6-b599-99b7de12645a None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.715264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b8a7a-34a9-4be6-b599-99b7de12645a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.716829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b8a7a-34a9-4be6-b599-99b7de12645a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:21.733451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 453/906] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:10:21 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:10:21.813874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84abb795-8784-4687-b8df-8f087ba6ab1c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:21.859189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa05d73-00ce-42dd-ba19-ac61e93f10b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:21.860216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa05d73-00ce-42dd-ba19-ac61e93f10b5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:21.860707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa05d73-00ce-42dd-ba19-ac61e93f10b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:21.861210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa05d73-00ce-42dd-ba19-ac61e93f10b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:21.874285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:21.940739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa05d73-00ce-42dd-ba19-ac61e93f10b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:21.942412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa05d73-00ce-42dd-ba19-ac61e93f10b5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:22.054069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 454/907] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:21 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:22.062261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b8a7a-34a9-4be6-b599-99b7de12645a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:22.173811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b8a7a-34a9-4be6-b599-99b7de12645a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:22.204400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:22.222871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:22.224352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:22.224856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:22.225452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:22.237521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:22.239583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:22.258151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:22.259149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:22.260081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:22.260081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:22.260081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:22.260081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:22.262412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 454/908] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:22.268365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa05d73-00ce-42dd-ba19-ac61e93f10b5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:22.312427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:22.377886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa05d73-00ce-42dd-ba19-ac61e93f10b5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:22.415338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97456569-cb30-4d4b-a5c2-0be6ae55378d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:22.416008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456569-cb30-4d4b-a5c2-0be6ae55378d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:22.416290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456569-cb30-4d4b-a5c2-0be6ae55378d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:22.416566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456569-cb30-4d4b-a5c2-0be6ae55378d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:22.417337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97456569-cb30-4d4b-a5c2-0be6ae55378d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:22.417646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97456569-cb30-4d4b-a5c2-0be6ae55378d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:22.419039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97456569-cb30-4d4b-a5c2-0be6ae55378d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:22.431422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 455/909] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:22.443372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:22.466155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721308-85c6-49d1-a785-64e9aeb37a25 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:22.466373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721308-85c6-49d1-a785-64e9aeb37a25 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:22.466543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0721308-85c6-49d1-a785-64e9aeb37a25 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:22.471871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 456/910] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:22.474829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a335bfd-f7c0-46bc-8d35-eb6df6e35c3f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:22.475038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a335bfd-f7c0-46bc-8d35-eb6df6e35c3f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:22.475274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a335bfd-f7c0-46bc-8d35-eb6df6e35c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:22.499063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0a335bfd-f7c0-46bc-8d35-eb6df6e35c3f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:22.500541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a335bfd-f7c0-46bc-8d35-eb6df6e35c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:22.614838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dff089-4f03-4f13-8127-92a9d688d09e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:22.616981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 457/911] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:22.652988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a335bfd-f7c0-46bc-8d35-eb6df6e35c3f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:22.653933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:22.654768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:22.655053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:22.655328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:22.662928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:22.664544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:22.679806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:22.680877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:22.681923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:22.681923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:22.681923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:22.681923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:22.681923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:22.681923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:22.681923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:22.681923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:22.686651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 455/912] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:22.687524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:22.725084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:22.727498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:22.800242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c694da-0496-4171-a6c1-4af1a2b1580f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:22.842382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:22.843443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:22.843939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:22.844423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:22.845469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:22.845995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:22.847636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:22.865699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 456/913] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:22.867786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:22.884465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a335bfd-f7c0-46bc-8d35-eb6df6e35c3f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:22.953162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:22.954442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:22.956539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:22.957024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:22.959238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:22.961013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:22.978913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:22.980048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:22.980931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 458/914] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:22.986429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:23.019771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:23.028203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f6b25-7754-4b9a-96c3-8b953d8af922 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:23.067573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:23.068648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.069214 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.069681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.070725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] RBAC: Adding query filter params (role.id=8a3e8dfadf1142a8adba3fe567890600, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=ae4e4bbf4359400289b9fd2ba8778067) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:23.071241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:23.072927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:23.085500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 457/915] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:22 2022] GET /identity/v3/role_assignments?user.id=ae4e4bbf4359400289b9fd2ba8778067&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=8a3e8dfadf1142a8adba3fe567890600 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:23.105655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:23.111915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a75765-f651-4a97-ad29-16198a351d5a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:23.150985 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e19de-9969-4562-ab71-6516bff65d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:23.151673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e19de-9969-4562-ab71-6516bff65d2a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.151952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e19de-9969-4562-ab71-6516bff65d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.152228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e19de-9969-4562-ab71-6516bff65d2a None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.152955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e19de-9969-4562-ab71-6516bff65d2a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:23.153300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e19de-9969-4562-ab71-6516bff65d2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:23.154700 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e19de-9969-4562-ab71-6516bff65d2a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:23.161234 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 459/916] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:23.166824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:23.283548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9688fc7-b0d6-43c9-acff-a24ce2065d2c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:23.301836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb72b02-cea7-4575-876c-af1c0dfed534 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.302357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb72b02-cea7-4575-876c-af1c0dfed534 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.302471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb72b02-cea7-4575-876c-af1c0dfed534 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.304072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 460/917] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:23.310504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a1cc8-dda1-41a0-be8a-9f5787eb9dd6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.311043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a1cc8-dda1-41a0-be8a-9f5787eb9dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.311521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a1cc8-dda1-41a0-be8a-9f5787eb9dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.319842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:23.320525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.320819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.321101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.321862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:23.322173 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:23.323577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:23.330683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d33a1cc8-dda1-41a0-be8a-9f5787eb9dd6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:23.332091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d33a1cc8-dda1-41a0-be8a-9f5787eb9dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:23.336644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 458/918] 10.222.0.66 () {56 vars in 1114 bytes} [Thu Jun 16 17:10:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:23.338525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:23.372506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:23.416742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 461/919] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:23.442907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a1cc8-dda1-41a0-be8a-9f5787eb9dd6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:23.471378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8229c584-81c2-4863-8fd3-cd892d8b7a95 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:23.512095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b285b-4f4a-4947-b16c-377c9bccf95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:23.513272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b285b-4f4a-4947-b16c-377c9bccf95a None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.514036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b285b-4f4a-4947-b16c-377c9bccf95a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.514909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b285b-4f4a-4947-b16c-377c9bccf95a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.516135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b285b-4f4a-4947-b16c-377c9bccf95a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:23.518755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b285b-4f4a-4947-b16c-377c9bccf95a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:23.548949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 459/920] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:10:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:23.556142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:23.573987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33a1cc8-dda1-41a0-be8a-9f5787eb9dd6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:23.623069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:23.624357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.624878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.625423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.626343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:23.628066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:23.642482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:23.643624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:23.644501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:23.647876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 462/921] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:23.655928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b285b-4f4a-4947-b16c-377c9bccf95a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:23.753086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:23.781080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b285b-4f4a-4947-b16c-377c9bccf95a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:23.816684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:23.817611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.817899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.818239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.818994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:23.819306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:23.820766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:23.826731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 460/922] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:23.832997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:23.939314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ac7a5-8ab1-406b-af0a-01c2c620b880 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:23.986229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4059e8-08bd-495c-abf2-12fba7d751d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:23.986914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4059e8-08bd-495c-abf2-12fba7d751d3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:23.987205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4059e8-08bd-495c-abf2-12fba7d751d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:23.987469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4059e8-08bd-495c-abf2-12fba7d751d3 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:23.988224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4059e8-08bd-495c-abf2-12fba7d751d3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:23.988545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4059e8-08bd-495c-abf2-12fba7d751d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:23.990262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4059e8-08bd-495c-abf2-12fba7d751d3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:23.996763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 463/923] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:24.003457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:24.066801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b101f1d-7fab-4232-a82e-f336f738d153 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:24.067085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b101f1d-7fab-4232-a82e-f336f738d153 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:24.067213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b101f1d-7fab-4232-a82e-f336f738d153 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:24.069037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 464/924] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:24.080000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0bc8-8333-4dd7-8b37-993d4d88a926 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:24.080403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0bc8-8333-4dd7-8b37-993d4d88a926 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:24.080780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0bc8-8333-4dd7-8b37-993d4d88a926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:24.103672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-502c0bc8-8333-4dd7-8b37-993d4d88a926 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:24.105594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-502c0bc8-8333-4dd7-8b37-993d4d88a926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:24.115783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:24.124909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6417f789-872c-4d76-b280-5b2e6d3100e6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:24.168214 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:24.169371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:24.169801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:24.170675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:24.171935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] RBAC: Adding query filter params (interface=public, service_id=d431cd23fe3144a19e1c15164d92baff, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:24.172411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:24.174224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:24.186666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 461/925] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Jun 16 17:10:24 2022] GET /identity/v3/endpoints?service_id=d431cd23fe3144a19e1c15164d92baff&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:24.187319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:24.206263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:24.208087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 465/926] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:24 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:24.233752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c0bc8-8333-4dd7-8b37-993d4d88a926 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:24.313087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0d3f0-7fba-4e61-881d-308a517acbbf None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:24.351264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4963a5-aff3-4f0f-b932-45d2d3d017b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:24.352003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4963a5-aff3-4f0f-b932-45d2d3d017b7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:24.352675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4963a5-aff3-4f0f-b932-45d2d3d017b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:24.353067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4963a5-aff3-4f0f-b932-45d2d3d017b7 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:24.353988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4963a5-aff3-4f0f-b932-45d2d3d017b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:24.355469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4963a5-aff3-4f0f-b932-45d2d3d017b7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:24.368840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 462/927] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:10:24 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:10:24.372595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c0bc8-8333-4dd7-8b37-993d4d88a926 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:24.416679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:24.417934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:24.418396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:24.418836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:24.419681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:24.421389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:24.435498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:24.436761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 17:10:24.437866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:24.440283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 466/928] 10.222.0.66 () {56 vars in 1132 bytes} [Thu Jun 16 17:10:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:24.447023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4963a5-aff3-4f0f-b932-45d2d3d017b7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:24.532782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:24.571448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4963a5-aff3-4f0f-b932-45d2d3d017b7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:24.608528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:24.609148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:24.609460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:24.609738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:24.610475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:24.610796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:24.612230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:24.618245 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 463/929] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:24.624530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:24.639312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:24.736808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a129bab6-bfcf-4d73-89e3-1e380431fac7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:24.772413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258c867-d14e-4924-97e0-fceaccdb1a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:24.773500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258c867-d14e-4924-97e0-fceaccdb1a9a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:24.773965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258c867-d14e-4924-97e0-fceaccdb1a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:24.774432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258c867-d14e-4924-97e0-fceaccdb1a9a None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:24.775334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258c867-d14e-4924-97e0-fceaccdb1a9a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:24.775852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258c867-d14e-4924-97e0-fceaccdb1a9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:24.777457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258c867-d14e-4924-97e0-fceaccdb1a9a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:24.783286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 467/930] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:24.789262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:24.901678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb8b3c-7a6a-4338-bff9-095f90dcf157 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:24.937798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d72da-3913-4084-a39a-6482cbc3e957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:24.938470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d72da-3913-4084-a39a-6482cbc3e957 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:24.938754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d72da-3913-4084-a39a-6482cbc3e957 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:24.939032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d72da-3913-4084-a39a-6482cbc3e957 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:24.939806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d72da-3913-4084-a39a-6482cbc3e957 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:24.941248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d72da-3913-4084-a39a-6482cbc3e957 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:24.945683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:24.965206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 464/931] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:10:24 2022] POST /identity/v3/endpoints => generated 347 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:25.352762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03333e-2070-4e61-8aee-c58819a738ea None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:25.353221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03333e-2070-4e61-8aee-c58819a738ea None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:25.353410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03333e-2070-4e61-8aee-c58819a738ea None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:25.355052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 468/932] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:25.360340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb79df-f984-47db-99d4-7844cc6a35b6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:25.360628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb79df-f984-47db-99d4-7844cc6a35b6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:25.360910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb79df-f984-47db-99d4-7844cc6a35b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:25.378947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cedb79df-f984-47db-99d4-7844cc6a35b6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:25.380185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cedb79df-f984-47db-99d4-7844cc6a35b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:25.461983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 465/933] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:25.484433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03333e-2070-4e61-8aee-c58819a738ea None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:25.566357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:25.604066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03333e-2070-4e61-8aee-c58819a738ea None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:25.640753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35da2c-f319-4952-8a4f-48aeb2328546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:25.642035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35da2c-f319-4952-8a4f-48aeb2328546 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:25.642525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35da2c-f319-4952-8a4f-48aeb2328546 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:25.643001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35da2c-f319-4952-8a4f-48aeb2328546 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:25.656443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35da2c-f319-4952-8a4f-48aeb2328546 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8b35da2c-f319-4952-8a4f-48aeb2328546 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:25.665242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:25.666227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:25.667127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:25.669984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 469/934] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Jun 16 17:10:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:25.681225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb79df-f984-47db-99d4-7844cc6a35b6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:25.778375 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb79df-f984-47db-99d4-7844cc6a35b6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:25.799170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:25.820731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1829f6cd-b7ef-43a5-8079-ace335cc9285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:25.821482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829f6cd-b7ef-43a5-8079-ace335cc9285 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:25.821764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829f6cd-b7ef-43a5-8079-ace335cc9285 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:25.822041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1829f6cd-b7ef-43a5-8079-ace335cc9285 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:25.822819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829f6cd-b7ef-43a5-8079-ace335cc9285 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:25.823130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829f6cd-b7ef-43a5-8079-ace335cc9285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:25.824541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1829f6cd-b7ef-43a5-8079-ace335cc9285 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:25.834241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 466/935] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Jun 16 17:10:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:25.839970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35da2c-f319-4952-8a4f-48aeb2328546 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:25.865203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6903-176d-4420-bb54-f73daca7783b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:25.865490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6903-176d-4420-bb54-f73daca7783b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:25.865695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc6903-176d-4420-bb54-f73daca7783b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:25.867264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 467/936] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:25.872518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d704c99-edf3-49ae-a663-a37d0c6b7380 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:25.872816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d704c99-edf3-49ae-a663-a37d0c6b7380 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:25.873076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d704c99-edf3-49ae-a663-a37d0c6b7380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:25.890114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1d704c99-edf3-49ae-a663-a37d0c6b7380 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:25.891315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1d704c99-edf3-49ae-a663-a37d0c6b7380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:25.977204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35da2c-f319-4952-8a4f-48aeb2328546 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:25.977716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 468/937] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:25.994801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d704c99-edf3-49ae-a663-a37d0c6b7380 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.002981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:26.011259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.012597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.012885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.013169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.027619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.029814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:26.053243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:26.054231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:26.055654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:26.055654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:26.055654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:26.055654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:26.055654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 470/938] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:26.060059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.112104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d704c99-edf3-49ae-a663-a37d0c6b7380 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:26.124008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:26.151577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.152716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.153232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.153702 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.154680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.156582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.158325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING keystone.server.flask.common [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 17:10:26.178431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.sql.core [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:10:26.178431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 17:10:26.178431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [parameters: {'id': '563fcb9cc3b24a87b953d9ba07b54d90', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 17:10:26.178431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80239) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 563fcb9cc3b24a87b953d9ba07b54d90.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 563fcb9cc3b24a87b953d9ba07b54d90. Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:10:26.187220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result.read() Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 17:10:26.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application yield resource Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise self.value Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application session.commit() Jun 16 17:10:26.189081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:10:26.190078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:10:26.191652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application result.read() Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:10:26.192693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [parameters: {'id': '563fcb9cc3b24a87b953d9ba07b54d90', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.194839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:26.201385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 563fcb9cc3b24a87b953d9ba07b54d90. Jun 16 17:10:26.203968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.206785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96654c-fc9f-40c3-974c-b0f1086478af None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:26.207013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 469/939] 10.222.0.66 () {60 vars in 1140 bytes} [Thu Jun 16 17:10:25 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:26.208871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.249081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.250434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.251770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.252242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.253165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:26.253690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.255557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.273040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 471/940] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:26.278471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.321927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b78e10-3cb7-4383-ad1b-494a1966f931 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:26.338867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:26.361640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.362419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.362826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.363125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.371990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.373814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:26.390049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:26.390990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.397208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 470/941] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:26.399479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.414002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41d171-c909-4366-867d-30f569ef2496 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:26.434640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:26.454862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.455889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.456112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.456391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.464596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.466489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:26.491356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:26.492650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:26.494036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 472/942] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:10:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:26.498056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.532121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8abac-a523-4ba1-b2d0-14a1de0b4b71 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:26.575909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f0fe85-606c-4df3-8210-44a27b398451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.576830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0fe85-606c-4df3-8210-44a27b398451 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.577309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0fe85-606c-4df3-8210-44a27b398451 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.578180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f0fe85-606c-4df3-8210-44a27b398451 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.579127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0fe85-606c-4df3-8210-44a27b398451 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:26.579580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0fe85-606c-4df3-8210-44a27b398451 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.581551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f0fe85-606c-4df3-8210-44a27b398451 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.617221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 471/943] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:26 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:26.620317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064b2b0-9f76-4d1b-97f4-c09b27562149 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:26.622385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:26.667496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.668245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.668451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24b51b-ce18-4670-a57a-194e3298ff32 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.668556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.668741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24b51b-ce18-4670-a57a-194e3298ff32 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.668857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.669012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24b51b-ce18-4670-a57a-194e3298ff32 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.669680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:26.669996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.670684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 472/944] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:26.671461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.675964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c97d1-c7cb-447c-b567-e9121229da35 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.676355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c97d1-c7cb-447c-b567-e9121229da35 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.676635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498c97d1-c7cb-447c-b567-e9121229da35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.696609 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-498c97d1-c7cb-447c-b567-e9121229da35 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:26.698174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-498c97d1-c7cb-447c-b567-e9121229da35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:26.701097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 473/945] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Jun 16 17:10:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:26.706657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.794639 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 473/946] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:26.814275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c97d1-c7cb-447c-b567-e9121229da35 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.833292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13219a40-df94-4c54-b6f6-50b2f47e8894 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:26.835865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:26.878572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.879005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.879308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.879583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.892310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.893803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:26.913198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:26.914188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:26.915112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:26.915112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:26.915112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:26.915112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:26.915352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 474/947] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:26.921034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:26.938305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:26.938998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498c97d1-c7cb-447c-b567-e9121229da35 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:26.976892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:26.978238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:26.978712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:26.979196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:26.980070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:26.981700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:26.994421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:26.995472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:26.996360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:26.999288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 474/948] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:27.005744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.032501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b220b-554c-4b35-8141-e7bfcc9bd15d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.072012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.072740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.073058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.073456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.074226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:27.074771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.076039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.090325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 475/949] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:27.095691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.097623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:27.118816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67956feb-539f-429e-8136-0bea871545c7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.159539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.160738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.161316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.161775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.162775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:27.163328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.165560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.177311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 475/950] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:27.183327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.199738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:27.240064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fe279-c0a8-4c98-b94f-3a7bfe21c9e0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.281580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e773ff-e43c-4ebe-9001-9347c27eb019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.282388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e773ff-e43c-4ebe-9001-9347c27eb019 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.282664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e773ff-e43c-4ebe-9001-9347c27eb019 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.282930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e773ff-e43c-4ebe-9001-9347c27eb019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.290192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e773ff-e43c-4ebe-9001-9347c27eb019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.291673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e773ff-e43c-4ebe-9001-9347c27eb019 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.295659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674de3f-a889-4df9-b9e1-8714bc41b9ca None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-97e773ff-e43c-4ebe-9001-9347c27eb019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:27.306879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:27.308041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:27.309123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 476/951] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:27.314182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4cc2-46da-410d-95aa-09cf5f1fb9d0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.314312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4cc2-46da-410d-95aa-09cf5f1fb9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.314500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4cc2-46da-410d-95aa-09cf5f1fb9d0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.316000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 477/952] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:27.316577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df4cc2-46da-410d-95aa-09cf5f1fb9d0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.340979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0fb92-4da0-49c9-ad8b-264cf3fd8a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.341696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0fb92-4da0-49c9-ad8b-264cf3fd8a59 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.341980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0fb92-4da0-49c9-ad8b-264cf3fd8a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.342267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0fb92-4da0-49c9-ad8b-264cf3fd8a59 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.342998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0fb92-4da0-49c9-ad8b-264cf3fd8a59 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:27.343291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0fb92-4da0-49c9-ad8b-264cf3fd8a59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.344660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0fb92-4da0-49c9-ad8b-264cf3fd8a59 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.351039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 476/953] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:27.352600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acc279-4158-4471-bfd0-d06bdd188712 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.352952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acc279-4158-4471-bfd0-d06bdd188712 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.353337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acc279-4158-4471-bfd0-d06bdd188712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.371887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-97acc279-4158-4471-bfd0-d06bdd188712 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:27.373571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-97acc279-4158-4471-bfd0-d06bdd188712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:27.375558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:27.425802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df4cc2-46da-410d-95aa-09cf5f1fb9d0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.452483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 477/954] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:27.453592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acc279-4158-4471-bfd0-d06bdd188712 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.466419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:27.475565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.476353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.476699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.477042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.478010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:27.478499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.480021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.494353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 478/955] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:27.499840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.574037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acc279-4158-4471-bfd0-d06bdd188712 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.607567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62c05f-f7e2-4cf8-93f3-a5ebb909f330 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.613788 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2ec17-bfdb-4f53-80e2-7d89416d4246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.614533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ec17-bfdb-4f53-80e2-7d89416d4246 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.614822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ec17-bfdb-4f53-80e2-7d89416d4246 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.615100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ec17-bfdb-4f53-80e2-7d89416d4246 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.615856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2ec17-bfdb-4f53-80e2-7d89416d4246 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.617262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2ec17-bfdb-4f53-80e2-7d89416d4246 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.625403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:27.645386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 478/956] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:10:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:27.647440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2ec17-bfdb-4f53-80e2-7d89416d4246 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.647993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.648969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.649431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.649765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.650570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.652128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:27.665814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:27.667248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:27.668382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:27.670773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 479/957] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:27 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:27.676566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.751896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:27.799076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2ec17-bfdb-4f53-80e2-7d89416d4246 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.817437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13226539-cc6e-4a1c-9cb4-6eb0abca00f2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:27.849990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d32338-5bcb-44e5-8960-731094397297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.851121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d32338-5bcb-44e5-8960-731094397297 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.851407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d32338-5bcb-44e5-8960-731094397297 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.851684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d32338-5bcb-44e5-8960-731094397297 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/ae4e4bbf4359400289b9fd2ba8778067/roles/8a3e8dfadf1142a8adba3fe567890600` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.854226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d32338-5bcb-44e5-8960-731094397297 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=ae4e4bbf4359400289b9fd2ba8778067, role_id=8a3e8dfadf1142a8adba3fe567890600)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.855904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d32338-5bcb-44e5-8960-731094397297 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.867845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d060bd-19fe-412b-840d-334d11dc062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:27.869020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d060bd-19fe-412b-840d-334d11dc062a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:27.869340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d060bd-19fe-412b-840d-334d11dc062a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:27.869614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d060bd-19fe-412b-840d-334d11dc062a None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:27.870432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d060bd-19fe-412b-840d-334d11dc062a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:27.870742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d060bd-19fe-412b-840d-334d11dc062a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:27.872188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d060bd-19fe-412b-840d-334d11dc062a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:27.874867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 479/958] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:10:27 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/ae4e4bbf4359400289b9fd2ba8778067/roles/8a3e8dfadf1142a8adba3fe567890600 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:10:27.883761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 480/959] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:27 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:27.890255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d32338-5bcb-44e5-8960-731094397297 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:27.954177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:28.006146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d32338-5bcb-44e5-8960-731094397297 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:28.048337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8303bc7-087e-435c-8b0a-24f9535510de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:28.049073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303bc7-087e-435c-8b0a-24f9535510de None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:28.049397 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303bc7-087e-435c-8b0a-24f9535510de None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:28.049679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303bc7-087e-435c-8b0a-24f9535510de None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:28.050489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8303bc7-087e-435c-8b0a-24f9535510de None admin] RBAC: Adding query filter params (type=compute) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:28.050803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8303bc7-087e-435c-8b0a-24f9535510de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:28.052659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8303bc7-087e-435c-8b0a-24f9535510de None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:28.058743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 480/960] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:27 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:28.913552 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc202ce2-c487-4c9f-9e35-f817805b82e7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:28.914056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc202ce2-c487-4c9f-9e35-f817805b82e7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:28.914056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc202ce2-c487-4c9f-9e35-f817805b82e7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:28.915768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 481/961] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:28.921281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa9c2-1930-4090-9736-6e5edef6b935 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:28.921719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa9c2-1930-4090-9736-6e5edef6b935 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:28.921886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa9c2-1930-4090-9736-6e5edef6b935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:28.939756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-41faa9c2-1930-4090-9736-6e5edef6b935 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:28.940969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-41faa9c2-1930-4090-9736-6e5edef6b935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:29.013623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 481/962] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:29.034493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc202ce2-c487-4c9f-9e35-f817805b82e7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:29.142892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc202ce2-c487-4c9f-9e35-f817805b82e7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:29.159488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:29.178116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:29.178719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:29.179000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:29.179273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:29.180021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:29.181511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:29.182872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING keystone.server.flask.common [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 17:10:29.197083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.sql.core [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:10:29.197083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 17:10:29.197083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [parameters: {'user_id': '2e96093fabfa40a880b03f93f1341607', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 17:10:29.197083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80238) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jun 16 17:10:29.214821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:10:29.216016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application yield resource Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise self.value Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application session.commit() Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 17:10:29.217027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.flush() Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application compat.raise_( Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 17:10:29.218002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 17:10:29.218953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 17:10:29.219840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [parameters: {'user_id': '2e96093fabfa40a880b03f93f1341607', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:29.220712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 17:10:29.226533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 17:10:29.227702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 17:10:29.227702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 17:10:29.227702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 17:10:29.227702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 17:10:29.227702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 17:10:29.227702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:29.227702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 482/963] 10.222.0.66 () {60 vars in 1131 bytes} [Thu Jun 16 17:10:29 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:29.228189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faa9c2-1930-4090-9736-6e5edef6b935 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:29.274480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:29.332471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faa9c2-1930-4090-9736-6e5edef6b935 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:29.369543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dac824-35a7-444b-8878-f85595207057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:29.370350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dac824-35a7-444b-8878-f85595207057 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:29.370635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dac824-35a7-444b-8878-f85595207057 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:29.370943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dac824-35a7-444b-8878-f85595207057 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:29.378364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dac824-35a7-444b-8878-f85595207057 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:29.380492 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dac824-35a7-444b-8878-f85595207057 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-93dac824-35a7-444b-8878-f85595207057 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:29.395902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:29.396877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:29.397884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 482/964] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:29.406655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:29.518668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acc781d-2fcb-49c9-ab92-2a4b5f46269d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:29.554730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea706c0-cbf0-464f-9629-4eed45de95ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:29.555446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea706c0-cbf0-464f-9629-4eed45de95ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:29.555730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea706c0-cbf0-464f-9629-4eed45de95ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:29.556006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea706c0-cbf0-464f-9629-4eed45de95ed None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:29.556830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea706c0-cbf0-464f-9629-4eed45de95ed None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:29.557137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea706c0-cbf0-464f-9629-4eed45de95ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:29.558845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea706c0-cbf0-464f-9629-4eed45de95ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:29.599047 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 483/965] 10.222.0.66 () {56 vars in 1101 bytes} [Thu Jun 16 17:10:29 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:29.882889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0abeaf-f6b1-4166-b04a-bca0480e0598 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:29.883079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0abeaf-f6b1-4166-b04a-bca0480e0598 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:29.883358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0abeaf-f6b1-4166-b04a-bca0480e0598 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:29.884914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 483/966] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:29.890228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b475-09f4-4de7-8786-85db2f19aecd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:29.890526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b475-09f4-4de7-8786-85db2f19aecd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:29.890806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188b475-09f4-4de7-8786-85db2f19aecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:29.911510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9188b475-09f4-4de7-8786-85db2f19aecd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:29.913057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9188b475-09f4-4de7-8786-85db2f19aecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:29.943588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:29.986587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 484/967] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:30.009103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0abeaf-f6b1-4166-b04a-bca0480e0598 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.106353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f891b9-cd24-46e5-b983-696a68d30535 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.106820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f891b9-cd24-46e5-b983-696a68d30535 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.106932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f891b9-cd24-46e5-b983-696a68d30535 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.108463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 485/968] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:30.114041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1439316-8ae3-4b13-8f75-68bcf4f91d1f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.114638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1439316-8ae3-4b13-8f75-68bcf4f91d1f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.115125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1439316-8ae3-4b13-8f75-68bcf4f91d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.124617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0abeaf-f6b1-4166-b04a-bca0480e0598 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.133058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c1439316-8ae3-4b13-8f75-68bcf4f91d1f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:30.134222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c1439316-8ae3-4b13-8f75-68bcf4f91d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:30.135868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:30.169695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.170656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.170990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.172098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.172776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:30.174170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:30.190342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:30.191466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:30.198417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 484/969] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:30.201574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.217733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 486/970] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:30.242252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1439316-8ae3-4b13-8f75-68bcf4f91d1f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.314418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5541e8df-6646-49df-b4e5-2d6e1fc7fe35 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.339212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:30.345490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1439316-8ae3-4b13-8f75-68bcf4f91d1f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.348548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f576419-f2b3-4362-a938-dd0babdf753f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.349235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576419-f2b3-4362-a938-dd0babdf753f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.349522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576419-f2b3-4362-a938-dd0babdf753f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.349799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f576419-f2b3-4362-a938-dd0babdf753f None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.350505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f576419-f2b3-4362-a938-dd0babdf753f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:30.350813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f576419-f2b3-4362-a938-dd0babdf753f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:30.352182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f576419-f2b3-4362-a938-dd0babdf753f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:30.356866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 485/971] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:30.358131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f86e2-669a-4dbd-95f5-eaf92c51a98b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.358416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f86e2-669a-4dbd-95f5-eaf92c51a98b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.358700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f86e2-669a-4dbd-95f5-eaf92c51a98b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.359947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 486/972] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:30.362888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f86e2-669a-4dbd-95f5-eaf92c51a98b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.385737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c7b1f-6607-4cf1-9eb0-64dfa8f4bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.386505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c7b1f-6607-4cf1-9eb0-64dfa8f4bf9d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.386788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c7b1f-6607-4cf1-9eb0-64dfa8f4bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.387096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c7b1f-6607-4cf1-9eb0-64dfa8f4bf9d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.398315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c7b1f-6607-4cf1-9eb0-64dfa8f4bf9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:30.401425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a29c7b1f-6607-4cf1-9eb0-64dfa8f4bf9d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:30.406979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:30.407965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:30.408838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 487/973] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:30.409611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c2023-59ef-40aa-bec2-e8b4fb4e3899 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.409701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c2023-59ef-40aa-bec2-e8b4fb4e3899 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.410118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c2023-59ef-40aa-bec2-e8b4fb4e3899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.426371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c02c2023-59ef-40aa-bec2-e8b4fb4e3899 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:30.427665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c02c2023-59ef-40aa-bec2-e8b4fb4e3899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:30.470218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f86e2-669a-4dbd-95f5-eaf92c51a98b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.500415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 488/974] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:30.501096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c2023-59ef-40aa-bec2-e8b4fb4e3899 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.505801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.506490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.506773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.507051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.507771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:30.508081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:30.509522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:30.514432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 487/975] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:30.520494 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.601546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:30.617621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c2023-59ef-40aa-bec2-e8b4fb4e3899 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.624009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:30.627450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5276bfee-e53f-44f9-850c-02704c024f49 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.653419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.654049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.654334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.654620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.655387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:30.655691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:30.657083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:30.661025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.661713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.661999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.662300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.663111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] RBAC: Adding query filter params (interface=admin, service_id=d431cd23fe3144a19e1c15164d92baff, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:30.663424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:30.664773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:30.665469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 489/976] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:30.666262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.670108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 488/977] 10.222.0.66 () {56 vars in 1249 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/endpoints?service_id=d431cd23fe3144a19e1c15164d92baff&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:30.671368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.767429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe60978-179b-4a7a-ad1b-f4dfa6b4a99a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.773773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f9906-d339-421e-a41b-e1e4bf1214df None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.800433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c68343-ac32-4850-8c35-2380089fe580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.801326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c68343-ac32-4850-8c35-2380089fe580 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.801691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c68343-ac32-4850-8c35-2380089fe580 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.801968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c68343-ac32-4850-8c35-2380089fe580 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.802687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c68343-ac32-4850-8c35-2380089fe580 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:30.804101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c68343-ac32-4850-8c35-2380089fe580 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:30.807607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.808461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.808938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.809409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.817936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 490/978] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:10:30 2022] POST /identity/v3/services => generated 242 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:30.818705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c68343-ac32-4850-8c35-2380089fe580 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.830703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:30.832347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:30.835100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:30.855108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:30.856314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:30.857369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:30.857369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:30.857369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:30.857369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:30.857369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 489/979] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:30.863078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:30.886465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:30.951199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c68343-ac32-4850-8c35-2380089fe580 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.985818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0435805-d52e-40f5-bbc0-c276b7b1936d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:30.994149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd488f9-6ef8-4979-861c-0cd692f5dfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:30.995227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd488f9-6ef8-4979-861c-0cd692f5dfaa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:30.995521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd488f9-6ef8-4979-861c-0cd692f5dfaa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:30.995803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd488f9-6ef8-4979-861c-0cd692f5dfaa None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:30.997570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd488f9-6ef8-4979-861c-0cd692f5dfaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:30.998293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd488f9-6ef8-4979-861c-0cd692f5dfaa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:31.006924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 491/980] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:10:31.022904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:31.023617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:31.023900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:31.024176 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:31.024946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:31.025329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:31.026808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:31.043581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 490/981] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:31.049680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd488f9-6ef8-4979-861c-0cd692f5dfaa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:31.130837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:31.172076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd488f9-6ef8-4979-861c-0cd692f5dfaa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:31.210111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:31.210864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:31.211168 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:31.211579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:31.220743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:31.222340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:31.238193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:31.239255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:31.240267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 492/982] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:10:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:31.246219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:31.325535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:31.350978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4945c-d851-43c0-b2a2-7b1a62eb7c3b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:31.386161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:31.386857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:31.387139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:31.387411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:31.388255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:31.388567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:31.390077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:31.408615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 491/983] 10.222.0.66 () {56 vars in 1149 bytes} [Thu Jun 16 17:10:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:31.414392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:31.523499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377c1c0-a0d2-4f62-a5ed-abaef0da28f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:31.549677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:31.558796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc03fa-8b2f-4394-b353-fbce837049bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:31.559449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc03fa-8b2f-4394-b353-fbce837049bd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:31.559728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc03fa-8b2f-4394-b353-fbce837049bd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:31.559997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cc03fa-8b2f-4394-b353-fbce837049bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:31.628270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc03fa-8b2f-4394-b353-fbce837049bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:31.629850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cc03fa-8b2f-4394-b353-fbce837049bd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:31.727951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 493/984] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:31 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:10:31.733511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:31.831026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac079d84-3308-438c-b672-5bd5d3cb9bd4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:31.864734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:31.865560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:31.865835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:31.866104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:31.867997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:31.877579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:31.879017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:31.897564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:31.898512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:31.899447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:31.899447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:31.899447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:31.899447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:31.901262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 492/985] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:31.904387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc03fa-8b2f-4394-b353-fbce837049bd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:31.934217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:32.013312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cc03fa-8b2f-4394-b353-fbce837049bd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:32.047703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:32.048373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:32.048653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:32.048927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:32.049682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:32.049991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:32.051410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:32.067689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 494/986] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:32.068857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:32.174516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f5cb8-8f3d-4fe8-b928-1170628a636c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:32.208835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:32.210102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:32.210367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:32.210659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:32.218136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:32.219574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:32.233135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:32.234074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:32.240246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 493/987] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:32.242052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:32.334962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:32.348205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a91f41-3a17-43f0-9cdb-141595dca635 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:32.382395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b95091-dc14-4959-ad02-c79cccb974b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:32.383058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b95091-dc14-4959-ad02-c79cccb974b4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:32.383343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b95091-dc14-4959-ad02-c79cccb974b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:32.383620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b95091-dc14-4959-ad02-c79cccb974b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:32.384407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b95091-dc14-4959-ad02-c79cccb974b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:32.384714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b95091-dc14-4959-ad02-c79cccb974b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:32.386183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b95091-dc14-4959-ad02-c79cccb974b4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:32.398825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 495/988] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:32.404715 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:32.452414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:32.516482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb32f32-205e-4f93-a21a-778e67267bc5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:32.552075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9814ad94-1c03-42d3-b418-dbeb60f141dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:32.552724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ad94-1c03-42d3-b418-dbeb60f141dd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:32.553006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ad94-1c03-42d3-b418-dbeb60f141dd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:32.553309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9814ad94-1c03-42d3-b418-dbeb60f141dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:32.554138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ad94-1c03-42d3-b418-dbeb60f141dd None admin] RBAC: Adding query filter params (role.id=8a3e8dfadf1142a8adba3fe567890600, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=ae4e4bbf4359400289b9fd2ba8778067) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:32.554450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ad94-1c03-42d3-b418-dbeb60f141dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:32.555873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9814ad94-1c03-42d3-b418-dbeb60f141dd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:32.562295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 494/989] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:32 2022] GET /identity/v3/role_assignments?user.id=ae4e4bbf4359400289b9fd2ba8778067&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=8a3e8dfadf1142a8adba3fe567890600 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:32.978712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c617a2-1572-48ba-aa28-34bd185f06ef None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:32.978962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c617a2-1572-48ba-aa28-34bd185f06ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:32.979252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c617a2-1572-48ba-aa28-34bd185f06ef None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:32.980792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 496/990] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:32.985856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f3f2-540d-49bd-b521-face7e9a2ec7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:32.986150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f3f2-540d-49bd-b521-face7e9a2ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:32.986407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7f3f2-540d-49bd-b521-face7e9a2ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.003964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7de7f3f2-540d-49bd-b521-face7e9a2ec7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:33.005403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7de7f3f2-540d-49bd-b521-face7e9a2ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:33.076980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 495/991] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:33.100896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c617a2-1572-48ba-aa28-34bd185f06ef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.144301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed2d75-201e-41d0-b5c1-b8a932e3ddb1 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.144632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed2d75-201e-41d0-b5c1-b8a932e3ddb1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.144895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed2d75-201e-41d0-b5c1-b8a932e3ddb1 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.146445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 496/992] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:33.151332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df5cca-cea7-4606-be99-4bee4c92e703 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.151608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df5cca-cea7-4606-be99-4bee4c92e703 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.151903 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df5cca-cea7-4606-be99-4bee4c92e703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.168813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-87df5cca-cea7-4606-be99-4bee4c92e703 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:33.170091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-87df5cca-cea7-4606-be99-4bee4c92e703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:33.204514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:33.215449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c617a2-1572-48ba-aa28-34bd185f06ef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.242916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 497/993] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:33 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:33.255137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.255941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.256219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.256486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.257135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.258667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.265328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df5cca-cea7-4606-be99-4bee4c92e703 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:33.270547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:33.271592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:33.277790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 497/994] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:33.278626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.295174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:33.379837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df5cca-cea7-4606-be99-4bee4c92e703 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.382784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cdd42f-6a63-48a1-a823-8e4b0c367273 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.416273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.416947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.417245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.417507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.418147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.418275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:33.418491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.419346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.419781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.419901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.420295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.421294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.423337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.424742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 498/995] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:33.430344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:33.437460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:33.438540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 17:10:33.439407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:33.440174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 498/996] 10.222.0.66 () {56 vars in 1126 bytes} [Thu Jun 16 17:10:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:33.445582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.484940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:33.516954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:33.543498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a86014-c071-4e16-9b5a-ccc3d4d819a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.556061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffcfe87-a588-47e0-9526-41edfb5d685f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.581389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.582064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.582457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.582732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.583785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:33.583976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.585757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.591105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 499/997] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:33.592710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.593403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.593684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.593956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.594664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:33.594967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.596561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.596662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.606444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 499/998] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:33.607754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.705668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44651c-fb0a-4604-a5c2-c7555fac8884 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.711425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:33.717038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f422f8bd-a646-4df3-ac54-45778e0eaad7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.742408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.743081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.743375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.743659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.744464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] RBAC: Adding query filter params (interface=public, service_id=8dc5b5d3f24f459cabd29c9b6f970156, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:33.744780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.746191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.756361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 500/999] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Jun 16 17:10:33 2022] GET /identity/v3/endpoints?service_id=8dc5b5d3f24f459cabd29c9b6f970156&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:33.757336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.759684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.760364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.760657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.760936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.761190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:33.762031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:33.762031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.763391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.773510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 500/1000] 10.222.0.66 () {56 vars in 1122 bytes} [Thu Jun 16 17:10:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:33.774591 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:33.867780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a7093-f457-48c4-aeb0-15ecd8df9343 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.877718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49adebbe-d8b8-4afa-8f12-069c586ee10c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:33.903482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a343cd-177c-4d9a-a0d9-23c242e89c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.904147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a343cd-177c-4d9a-a0d9-23c242e89c27 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.904446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a343cd-177c-4d9a-a0d9-23c242e89c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.904719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a343cd-177c-4d9a-a0d9-23c242e89c27 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.905539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a343cd-177c-4d9a-a0d9-23c242e89c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.907033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a343cd-177c-4d9a-a0d9-23c242e89c27 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.910770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cf7aa-461f-4b89-8f52-41ffcc0fd1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:33.911423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf7aa-461f-4b89-8f52-41ffcc0fd1db None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:33.911704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf7aa-461f-4b89-8f52-41ffcc0fd1db None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:33.911975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cf7aa-461f-4b89-8f52-41ffcc0fd1db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:33.912706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cf7aa-461f-4b89-8f52-41ffcc0fd1db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:33.913800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 501/1001] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:10:33 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:10:33.914076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cf7aa-461f-4b89-8f52-41ffcc0fd1db None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:33.933338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 501/1002] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:10:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:34.687285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f235bb-6f63-469e-adc3-763f194ef729 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:34.687816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f235bb-6f63-469e-adc3-763f194ef729 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:34.687816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f235bb-6f63-469e-adc3-763f194ef729 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:34.689433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 502/1003] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:34.694594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2def1a0-1643-40d0-a17a-de264c1986be None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:34.695036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2def1a0-1643-40d0-a17a-de264c1986be None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:34.695182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2def1a0-1643-40d0-a17a-de264c1986be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:34.713364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a2def1a0-1643-40d0-a17a-de264c1986be None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:34.714678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a2def1a0-1643-40d0-a17a-de264c1986be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:34.734742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:34.793581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 502/1004] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:34 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:34.815606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f235bb-6f63-469e-adc3-763f194ef729 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:34.886756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:34.922352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f235bb-6f63-469e-adc3-763f194ef729 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:34.955615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:34.956411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:34.956691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:34.956962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:34.957911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:34.959278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:34.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:34.972206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:34.973137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:34.977276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 503/1005] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Jun 16 17:10:34 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:34.978530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2def1a0-1643-40d0-a17a-de264c1986be None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:35.087380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2def1a0-1643-40d0-a17a-de264c1986be None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:35.121515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9e2f4-11bd-4b49-9636-ca2e489a5ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:35.122192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9e2f4-11bd-4b49-9636-ca2e489a5ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:35.122471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9e2f4-11bd-4b49-9636-ca2e489a5ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:35.122748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9e2f4-11bd-4b49-9636-ca2e489a5ea1 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:35.123460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9e2f4-11bd-4b49-9636-ca2e489a5ea1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:35.123766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9e2f4-11bd-4b49-9636-ca2e489a5ea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:35.125132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9e2f4-11bd-4b49-9636-ca2e489a5ea1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:35.134645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 503/1006] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:35.136174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:35.239241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27530ac5-a6b1-4b04-bcdf-00d4403aee02 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:35.267844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:35.273856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f4aa9-67ee-4cce-a6ab-5a5294355ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:35.274526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f4aa9-67ee-4cce-a6ab-5a5294355ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:35.274821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f4aa9-67ee-4cce-a6ab-5a5294355ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:35.275076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f4aa9-67ee-4cce-a6ab-5a5294355ba7 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:35.275774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f4aa9-67ee-4cce-a6ab-5a5294355ba7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:35.276074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f4aa9-67ee-4cce-a6ab-5a5294355ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:35.277471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f4aa9-67ee-4cce-a6ab-5a5294355ba7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:35.282188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 504/1007] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:36.041917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003300a2-c168-466f-88c7-ae60468d0c2b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.042462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003300a2-c168-466f-88c7-ae60468d0c2b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.042462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003300a2-c168-466f-88c7-ae60468d0c2b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.044021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 504/1008] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:36.049083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced80c3-768b-4400-8f0a-ca22b4fb5277 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.049527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced80c3-768b-4400-8f0a-ca22b4fb5277 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.049728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ced80c3-768b-4400-8f0a-ca22b4fb5277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.061932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec69c20-9b12-48e7-8d8a-2cb240586df9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.062223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec69c20-9b12-48e7-8d8a-2cb240586df9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.062491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec69c20-9b12-48e7-8d8a-2cb240586df9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.064015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 505/1009] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:36.066630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0ced80c3-768b-4400-8f0a-ca22b4fb5277 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:36.067841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ced80c3-768b-4400-8f0a-ca22b4fb5277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:36.068696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101009-ca9f-4893-bb3d-0285d084574b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.068959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101009-ca9f-4893-bb3d-0285d084574b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.069285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b101009-ca9f-4893-bb3d-0285d084574b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.088216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1b101009-ca9f-4893-bb3d-0285d084574b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:36.089422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1b101009-ca9f-4893-bb3d-0285d084574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:36.097254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:36.142506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 505/1010] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:36.159809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 506/1011] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:36 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:36.164922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced80c3-768b-4400-8f0a-ca22b4fb5277 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:36.182647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b101009-ca9f-4893-bb3d-0285d084574b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:36.264193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ced80c3-768b-4400-8f0a-ca22b4fb5277 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:36.285343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b101009-ca9f-4893-bb3d-0285d084574b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:36.290914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:36.300288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:36.301050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.301379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.301722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.302408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:36.303793 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:36.315735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:36.316968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 17:10:36.317884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:36.322436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 506/1012] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:36.323140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:36.334007 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b7409-8f98-4c6b-a5d8-2cb9b60f6a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:36.335183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b7409-8f98-4c6b-a5d8-2cb9b60f6a2a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.335414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b7409-8f98-4c6b-a5d8-2cb9b60f6a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.335799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b7409-8f98-4c6b-a5d8-2cb9b60f6a2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.337092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b7409-8f98-4c6b-a5d8-2cb9b60f6a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:36.338722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582b7409-8f98-4c6b-a5d8-2cb9b60f6a2a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:36.341220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 507/1013] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:36.346408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b7409-8f98-4c6b-a5d8-2cb9b60f6a2a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:36.362312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:36.427911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b15d2-09d7-4433-a467-b165d9896fc8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:36.457344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582b7409-8f98-4c6b-a5d8-2cb9b60f6a2a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:36.462972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:36.463647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.463930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.464205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.464912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:36.465243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:36.466646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:36.477237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 507/1014] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:36.478786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:36.506604 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e41a2-75c4-49f4-9433-faa9243edfc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:36.507427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e41a2-75c4-49f4-9433-faa9243edfc0 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.507711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e41a2-75c4-49f4-9433-faa9243edfc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.507986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e41a2-75c4-49f4-9433-faa9243edfc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.508696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e41a2-75c4-49f4-9433-faa9243edfc0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:36.510176 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e41a2-75c4-49f4-9433-faa9243edfc0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:36.538741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 508/1015] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:10:36 2022] POST /identity/v3/projects => generated 311 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:10:36.539640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:36.598938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e3512-5e6d-4ade-a96b-cfb19cf4e8c2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:36.635459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fceb3-aa7b-48c6-a85c-43bf6308e0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:36.636144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fceb3-aa7b-48c6-a85c-43bf6308e0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.636428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fceb3-aa7b-48c6-a85c-43bf6308e0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.636705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fceb3-aa7b-48c6-a85c-43bf6308e0f8 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.637456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fceb3-aa7b-48c6-a85c-43bf6308e0f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:36.637766 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fceb3-aa7b-48c6-a85c-43bf6308e0f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:36.639149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fceb3-aa7b-48c6-a85c-43bf6308e0f8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:36.644950 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 508/1016] 10.222.0.66 () {56 vars in 1112 bytes} [Thu Jun 16 17:10:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:36.650931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e41a2-75c4-49f4-9433-faa9243edfc0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:36.717117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:36.760360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e41a2-75c4-49f4-9433-faa9243edfc0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:36.794351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e602c7f9-7535-4277-a1d3-ea39174f8d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:36.795019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602c7f9-7535-4277-a1d3-ea39174f8d58 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:36.795309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602c7f9-7535-4277-a1d3-ea39174f8d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:36.795591 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e602c7f9-7535-4277-a1d3-ea39174f8d58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:36.796344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602c7f9-7535-4277-a1d3-ea39174f8d58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:36.797755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e602c7f9-7535-4277-a1d3-ea39174f8d58 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:36.815833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 509/1017] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:10:36 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:37.365772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24177f-b54c-4cef-9df3-83d1ddb2b976 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:37.366298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24177f-b54c-4cef-9df3-83d1ddb2b976 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:37.366377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24177f-b54c-4cef-9df3-83d1ddb2b976 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:37.367736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 509/1018] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:37.373054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481add-5960-4dda-a435-5b4c28ba2dbd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:37.373546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481add-5960-4dda-a435-5b4c28ba2dbd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:37.373627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c481add-5960-4dda-a435-5b4c28ba2dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:37.391328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0c481add-5960-4dda-a435-5b4c28ba2dbd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:37.392593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0c481add-5960-4dda-a435-5b4c28ba2dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:37.467696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 510/1019] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:37 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:37.490149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24177f-b54c-4cef-9df3-83d1ddb2b976 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:37.598943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24177f-b54c-4cef-9df3-83d1ddb2b976 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:37.618366 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:37.637551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01be73-f8bd-4f5d-bf2a-e3e1ce20ee80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:37.638221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01be73-f8bd-4f5d-bf2a-e3e1ce20ee80 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:37.638501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01be73-f8bd-4f5d-bf2a-e3e1ce20ee80 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:37.638774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01be73-f8bd-4f5d-bf2a-e3e1ce20ee80 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:37.639502 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01be73-f8bd-4f5d-bf2a-e3e1ce20ee80 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:37.640868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01be73-f8bd-4f5d-bf2a-e3e1ce20ee80 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:37.660195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 510/1020] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:10:37 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:38.712662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179740f-3893-41b9-8be7-a59ef0c4a884 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:38.713423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179740f-3893-41b9-8be7-a59ef0c4a884 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:38.713423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1179740f-3893-41b9-8be7-a59ef0c4a884 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:38.714785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 511/1021] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:38.719899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6990e-0fe6-4e9b-beda-9026d961c417 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:38.720309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6990e-0fe6-4e9b-beda-9026d961c417 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:38.720478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6990e-0fe6-4e9b-beda-9026d961c417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:38.740313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8ae6990e-0fe6-4e9b-beda-9026d961c417 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:38.741446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8ae6990e-0fe6-4e9b-beda-9026d961c417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:38.819373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 511/1022] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:38.843818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179740f-3893-41b9-8be7-a59ef0c4a884 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:38.893370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:38.959792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1179740f-3893-41b9-8be7-a59ef0c4a884 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:38.983676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45871eb-f0c5-4b36-81d7-4cd6663a04b0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:38.984198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45871eb-f0c5-4b36-81d7-4cd6663a04b0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:38.984705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45871eb-f0c5-4b36-81d7-4cd6663a04b0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:38.986149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 512/1023] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:38.991523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2329a-ac43-4c71-a6fc-e25444020282 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:38.991960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2329a-ac43-4c71-a6fc-e25444020282 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:38.992242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd2329a-ac43-4c71-a6fc-e25444020282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:38.997701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bde2d7-0682-4015-946e-308b811fe2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:38.998509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bde2d7-0682-4015-946e-308b811fe2ff None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:38.998793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bde2d7-0682-4015-946e-308b811fe2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:38.999071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bde2d7-0682-4015-946e-308b811fe2ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.000264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bde2d7-0682-4015-946e-308b811fe2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:39.001744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bde2d7-0682-4015-946e-308b811fe2ff None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:39.005326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 512/1024] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:39.008889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bde2d7-0682-4015-946e-308b811fe2ff None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:39.010580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5bd2329a-ac43-4c71-a6fc-e25444020282 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:39.011959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5bd2329a-ac43-4c71-a6fc-e25444020282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:39.089219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 513/1025] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:39.110536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2329a-ac43-4c71-a6fc-e25444020282 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:39.110984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bde2d7-0682-4015-946e-308b811fe2ff None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:39.124665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:39.148994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e124bd-d8a5-4993-8f53-9ce6337474d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:39.149910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e124bd-d8a5-4993-8f53-9ce6337474d0 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:39.150195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e124bd-d8a5-4993-8f53-9ce6337474d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:39.150493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e124bd-d8a5-4993-8f53-9ce6337474d0 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.151174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e124bd-d8a5-4993-8f53-9ce6337474d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:39.151850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:39.153152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e124bd-d8a5-4993-8f53-9ce6337474d0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:39.179281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 513/1026] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:10:39 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:39.231031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd2329a-ac43-4c71-a6fc-e25444020282 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:39.269076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c4af-50b5-4f45-b774-170e9bf2f3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:39.269921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c4af-50b5-4f45-b774-170e9bf2f3fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:39.270200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c4af-50b5-4f45-b774-170e9bf2f3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:39.270475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0c4af-50b5-4f45-b774-170e9bf2f3fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.282465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0c4af-50b5-4f45-b774-170e9bf2f3fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b6c0c4af-50b5-4f45-b774-170e9bf2f3fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:39.290876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:39.291995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:39.292895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 514/1027] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Jun 16 17:10:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:39.298494 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e124bd-d8a5-4993-8f53-9ce6337474d0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:39.409781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e124bd-d8a5-4993-8f53-9ce6337474d0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:39.445751 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:39.446430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:39.446709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:39.446981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.447739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:39.448048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:39.449451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:39.458447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 514/1028] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Jun 16 17:10:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:39.464621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c4af-50b5-4f45-b774-170e9bf2f3fe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:39.564039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:39.572872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0c4af-50b5-4f45-b774-170e9bf2f3fe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:39.609514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc01cda6-7845-491d-aa59-3358a1d7c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:39.610320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01cda6-7845-491d-aa59-3358a1d7c394 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:39.610601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01cda6-7845-491d-aa59-3358a1d7c394 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:39.610875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc01cda6-7845-491d-aa59-3358a1d7c394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.622008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01cda6-7845-491d-aa59-3358a1d7c394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:39.623466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc01cda6-7845-491d-aa59-3358a1d7c394 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bc01cda6-7845-491d-aa59-3358a1d7c394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:39.642454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:39.643397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:39.644327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:39.644327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:39.644327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:39.644327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:39.649267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 515/1029] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:39.651203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:39.685234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:39.762953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40bafe-ba1a-4c0f-81f0-67cbea8b01d9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:39.763480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40bafe-ba1a-4c0f-81f0-67cbea8b01d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:39.763658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40bafe-ba1a-4c0f-81f0-67cbea8b01d9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.765092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 516/1030] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:39.767728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f99973-9fcd-4fa5-97d5-8c0f96cad28f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:39.770418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382fc70-e74a-41ba-8b71-0a561f7d9334 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:39.770778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382fc70-e74a-41ba-8b71-0a561f7d9334 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:39.771129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b382fc70-e74a-41ba-8b71-0a561f7d9334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.788288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b382fc70-e74a-41ba-8b71-0a561f7d9334 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:39.789517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b382fc70-e74a-41ba-8b71-0a561f7d9334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:39.804305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:39.804977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:39.805289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:39.805569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.806295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:39.806664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:39.808042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:39.820739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 515/1031] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:39.826101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:39.860563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 517/1032] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:39.883577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382fc70-e74a-41ba-8b71-0a561f7d9334 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:39.923725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:39.935084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654576a4-24a9-4956-88ef-ea3b1efdf6f2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:39.971064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:39.973944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:39.974798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:39.975242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:39.975548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:39.984140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:39.986029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:39.995450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b382fc70-e74a-41ba-8b71-0a561f7d9334 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:40.001544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:40.002493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:40.003484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 516/1033] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:10:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:40.008635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.034865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.035811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.036091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.036361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.037016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.038437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:40.050266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:40.051488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:40.052371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 518/1034] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Jun 16 17:10:39 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:40.057759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.118768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762329c-3c0a-48ee-a1ee-754909a5a985 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.154775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.155451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.155734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.156019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.156848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:40.157158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.158613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.164062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2992f0-0636-4503-b883-e1514a8afb10 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.176642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 517/1035] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:40.182008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.192966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:40.204258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.205364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.205855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.206345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.207448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] RBAC: Adding query filter params (name=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:40.208014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.210012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.217126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 519/1036] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:40.222447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.226948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:40.291403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f412d6-944c-4faf-8b86-122418055acb None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.328888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbece380-6be4-45ba-b22f-d1c51cf71ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.329620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbece380-6be4-45ba-b22f-d1c51cf71ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.329933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbece380-6be4-45ba-b22f-d1c51cf71ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.330232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbece380-6be4-45ba-b22f-d1c51cf71ff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.333710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9999c8d-986e-4c7a-8146-792c7d822723 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.372797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.373505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.373708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.373955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.374661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:40.374964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.376338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.381923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 520/1037] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:40.387228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.407419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbece380-6be4-45ba-b22f-d1c51cf71ff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.408871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbece380-6be4-45ba-b22f-d1c51cf71ff5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.419652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:40.460712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:40.502586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba1e47-2fd0-494b-8705-af513e484d6f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.518310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 518/1038] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:10:40.523905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbece380-6be4-45ba-b22f-d1c51cf71ff5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.540324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.541080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.541470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.541754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.542558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] RBAC: Adding query filter params (interface=public, service_id=57253d0cdfee4e10a23ecada559e78e8, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:40.542880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.544280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.554983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 521/1039] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/endpoints?service_id=57253d0cdfee4e10a23ecada559e78e8&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:40.555983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.633574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbece380-6be4-45ba-b22f-d1c51cf71ff5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.642802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:40.658246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9cb3e-290e-481f-9046-a9d976399097 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.669998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.670803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.671084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.671361 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.681311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.682755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.688628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:40.695549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1851b39-b4ff-4cc2-b634-a967f42a8f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.696212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851b39-b4ff-4cc2-b634-a967f42a8f36 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.696494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851b39-b4ff-4cc2-b634-a967f42a8f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.696769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1851b39-b4ff-4cc2-b634-a967f42a8f36 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.697621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1851b39-b4ff-4cc2-b634-a967f42a8f36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.698988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1851b39-b4ff-4cc2-b634-a967f42a8f36 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:40.700928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:40.701983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:40.702908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:40.702908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:40.702908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:40.702908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:40.702908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 519/1040] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:40.717694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 522/1041] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:10:40.717911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.826771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9555cb4f-4686-44da-858e-642f6518fabc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:40.864066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:40.864801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:40.865089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:40.865396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:40.866132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:40.866886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:40.867813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:40.882195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 520/1042] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:40.887717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1851b39-b4ff-4cc2-b634-a967f42a8f36 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:40.993242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1851b39-b4ff-4cc2-b634-a967f42a8f36 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:41.029281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:41.030075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:41.030352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:41.030625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:41.037827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:41.039268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:41.053682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:41.054621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:41.055550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 523/1043] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:41.062284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:41.087714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:41.172353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06d96a-127e-4fc9-aa39-a482a435bb6b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:41.209023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca7ffae-155d-4ca4-8254-1d8a8e78f28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:41.210046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7ffae-155d-4ca4-8254-1d8a8e78f28e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:41.210334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7ffae-155d-4ca4-8254-1d8a8e78f28e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:41.210615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca7ffae-155d-4ca4-8254-1d8a8e78f28e None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:41.211445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7ffae-155d-4ca4-8254-1d8a8e78f28e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:41.211784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7ffae-155d-4ca4-8254-1d8a8e78f28e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:41.213760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca7ffae-155d-4ca4-8254-1d8a8e78f28e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:41.227215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 521/1044] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:41.233053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:41.296771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:41.350199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6327d-1f66-4b86-b715-95f90bd51db1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:41.356618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69af231-f948-481c-a41e-56b939e9ce9e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:41.356870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69af231-f948-481c-a41e-56b939e9ce9e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:41.357214 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69af231-f948-481c-a41e-56b939e9ce9e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:41.358737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 522/1045] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:41.364002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b64728-aa46-4b82-92d9-8614a01a488a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:41.364408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b64728-aa46-4b82-92d9-8614a01a488a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:41.364527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b64728-aa46-4b82-92d9-8614a01a488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:41.381910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-56b64728-aa46-4b82-92d9-8614a01a488a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:41.383366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-56b64728-aa46-4b82-92d9-8614a01a488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:41.393111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:41.393797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:41.394387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:41.394703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:41.395714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] RBAC: Adding query filter params (role.id=8a3e8dfadf1142a8adba3fe567890600, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=e3241452a88a40da9c9b8fa79c4fc9c0) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:41.396026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:41.397480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:41.406710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 524/1046] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:41 2022] GET /identity/v3/role_assignments?user.id=e3241452a88a40da9c9b8fa79c4fc9c0&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=8a3e8dfadf1142a8adba3fe567890600 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:41.466584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 523/1047] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:41.490063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:41.603477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0bcb99-8227-4752-a91c-c5f9d6145116 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:41.626106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:41.642284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606a43e-964f-414a-a3ac-00fc0bc40782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:41.643102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a43e-964f-414a-a3ac-00fc0bc40782 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:41.643371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a43e-964f-414a-a3ac-00fc0bc40782 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:41.643631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a43e-964f-414a-a3ac-00fc0bc40782 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:41.654859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606a43e-964f-414a-a3ac-00fc0bc40782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8606a43e-964f-414a-a3ac-00fc0bc40782 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:41.663066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:41.664005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:41.664873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 525/1048] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:41.670475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b64728-aa46-4b82-92d9-8614a01a488a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:41.682125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:41.775035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b64728-aa46-4b82-92d9-8614a01a488a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:41.809768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:41.810434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:41.810715 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:41.810984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:41.811733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:41.812039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:41.813457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:41.823413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 524/1049] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:41.829229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606a43e-964f-414a-a3ac-00fc0bc40782 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:41.936815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606a43e-964f-414a-a3ac-00fc0bc40782 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:41.972066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a1bd9-ef0e-4622-8aa9-bdc90deae11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:41.973010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1bd9-ef0e-4622-8aa9-bdc90deae11b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:41.973277 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1bd9-ef0e-4622-8aa9-bdc90deae11b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:41.973568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7a1bd9-ef0e-4622-8aa9-bdc90deae11b None admin] PATH_INFO: `/v3/users/673614eb4ff64bd29279b95016e6654f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:41.987247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a1bd9-ef0e-4622-8aa9-bdc90deae11b None admin] RBAC: Authorizing `identity:get_user(user_id=673614eb4ff64bd29279b95016e6654f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:41.988764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7a1bd9-ef0e-4622-8aa9-bdc90deae11b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:41.990629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 526/1050] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:10:41 2022] GET /identity/v3/users/673614eb4ff64bd29279b95016e6654f => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:41.996490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:42.067298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:42.099148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edb0f4-18e0-4937-9fec-31b627f38964 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:42.134890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d07ef-3653-4179-a4be-113c180ab675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:42.135542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d07ef-3653-4179-a4be-113c180ab675 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:42.135824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d07ef-3653-4179-a4be-113c180ab675 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:42.136100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d07ef-3653-4179-a4be-113c180ab675 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:42.204973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d07ef-3653-4179-a4be-113c180ab675 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:42.206470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d07ef-3653-4179-a4be-113c180ab675 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:42.275198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:42.307154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 525/1051] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:42.312855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a1bd9-ef0e-4622-8aa9-bdc90deae11b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:42.356505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:42.419677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7a1bd9-ef0e-4622-8aa9-bdc90deae11b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:42.453555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f35e2-7726-4fbb-84dc-e48538f0f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:42.454472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f35e2-7726-4fbb-84dc-e48538f0f1de None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:42.454746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f35e2-7726-4fbb-84dc-e48538f0f1de None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:42.455155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f35e2-7726-4fbb-84dc-e48538f0f1de None admin] PATH_INFO: `/v3/projects/c6b34fe332ce4538b6729042368ac539` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:42.456227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f35e2-7726-4fbb-84dc-e48538f0f1de None admin] RBAC: Authorizing `identity:get_project(project_id=c6b34fe332ce4538b6729042368ac539)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:42.457938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835f35e2-7726-4fbb-84dc-e48538f0f1de None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:42.464869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 527/1052] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:10:42 2022] GET /identity/v3/projects/c6b34fe332ce4538b6729042368ac539 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:42.466028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d07ef-3653-4179-a4be-113c180ab675 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:42.573788 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d07ef-3653-4179-a4be-113c180ab675 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:42.609906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf869595-0348-45a5-bf53-56dec072fa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:42.610470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869595-0348-45a5-bf53-56dec072fa5c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:42.610753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869595-0348-45a5-bf53-56dec072fa5c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:42.611033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf869595-0348-45a5-bf53-56dec072fa5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:42.611870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869595-0348-45a5-bf53-56dec072fa5c None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=c6b34fe332ce4538b6729042368ac539, user.id=673614eb4ff64bd29279b95016e6654f) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:42.612185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869595-0348-45a5-bf53-56dec072fa5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:42.614290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf869595-0348-45a5-bf53-56dec072fa5c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:42.629276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 526/1053] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:42 2022] GET /identity/v3/role_assignments?user.id=673614eb4ff64bd29279b95016e6654f&scope.project.id=c6b34fe332ce4538b6729042368ac539&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:42.847585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366253f-f75a-47ff-b77b-d29304a600fc None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:42.847937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366253f-f75a-47ff-b77b-d29304a600fc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:42.848099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366253f-f75a-47ff-b77b-d29304a600fc None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:42.849768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 528/1054] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:42.854992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887f35-dae5-46a9-8876-3b2a3542b1e6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:42.855281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887f35-dae5-46a9-8876-3b2a3542b1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:42.855554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e887f35-dae5-46a9-8876-3b2a3542b1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:42.873458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1e887f35-dae5-46a9-8876-3b2a3542b1e6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:42.874570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1e887f35-dae5-46a9-8876-3b2a3542b1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:42.907035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:42.949555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 527/1055] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:42.971900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366253f-f75a-47ff-b77b-d29304a600fc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:43.075637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366253f-f75a-47ff-b77b-d29304a600fc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:43.083471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:43.110997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:43.111799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:43.112079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:43.112353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:43.113005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:43.114449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:43.126525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:43.127528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 17:10:43.128398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:43.129996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 529/1056] 10.222.0.66 () {56 vars in 1105 bytes} [Thu Jun 16 17:10:42 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:43.134052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e887f35-dae5-46a9-8876-3b2a3542b1e6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:43.236357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e887f35-dae5-46a9-8876-3b2a3542b1e6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:43.270820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:43.271523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:43.271805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:43.272074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:43.272781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] RBAC: Adding query filter params (name=image) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:43.273083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:43.274473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:43.284124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 528/1057] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:43.285523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:43.391923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821914e-4d11-48d2-832e-03b2b133d9e6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:43.426768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2026264-d1a1-4441-bc03-e78ab9f7bfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:43.427433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026264-d1a1-4441-bc03-e78ab9f7bfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:43.427713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026264-d1a1-4441-bc03-e78ab9f7bfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:43.427988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026264-d1a1-4441-bc03-e78ab9f7bfa3 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:43.428695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2026264-d1a1-4441-bc03-e78ab9f7bfa3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:43.429004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2026264-d1a1-4441-bc03-e78ab9f7bfa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:43.430404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2026264-d1a1-4441-bc03-e78ab9f7bfa3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:43.435573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 530/1058] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:43.441200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:43.476616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:43.556388 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e8565-4e3c-4e96-9629-81b9d1be8bec None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:43.569809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1df14-eb30-4857-8ee8-843126d9958a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:43.570090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1df14-eb30-4857-8ee8-843126d9958a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:43.570369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1df14-eb30-4857-8ee8-843126d9958a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:43.571745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 531/1059] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:10:43.576627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5128b-60ca-4baa-b65b-cf982ab278c2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:43.576910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5128b-60ca-4baa-b65b-cf982ab278c2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:43.577187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5128b-60ca-4baa-b65b-cf982ab278c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:43.592514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d4d1b-bba6-4f0b-b6b6-e46df18be36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:43.593195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d4d1b-bba6-4f0b-b6b6-e46df18be36a None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:43.593488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d4d1b-bba6-4f0b-b6b6-e46df18be36a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:43.593765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d4d1b-bba6-4f0b-b6b6-e46df18be36a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:43.593997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f7a5128b-60ca-4baa-b65b-cf982ab278c2 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:43.594516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d4d1b-bba6-4f0b-b6b6-e46df18be36a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:43.595180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7a5128b-60ca-4baa-b65b-cf982ab278c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:43.595904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d4d1b-bba6-4f0b-b6b6-e46df18be36a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:43.615608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 529/1060] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:10:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:43.625625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:43.680290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 532/1061] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:43.703217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d4d1b-bba6-4f0b-b6b6-e46df18be36a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:43.799333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:43.817617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d4d1b-bba6-4f0b-b6b6-e46df18be36a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:43.852501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b63555-ed6b-42c1-b102-e57ed87fd8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:43.853360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b63555-ed6b-42c1-b102-e57ed87fd8ce None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:43.853646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b63555-ed6b-42c1-b102-e57ed87fd8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:43.853965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b63555-ed6b-42c1-b102-e57ed87fd8ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:43.864677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b63555-ed6b-42c1-b102-e57ed87fd8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f6b63555-ed6b-42c1-b102-e57ed87fd8ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:43.872822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:43.873917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:43.874783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 530/1062] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Jun 16 17:10:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:43.880574 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5128b-60ca-4baa-b65b-cf982ab278c2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:43.982782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5128b-60ca-4baa-b65b-cf982ab278c2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:44.017453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:44.018132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:44.018411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:44.018681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:44.019491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:44.019851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:44.021363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:44.029782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 533/1063] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Jun 16 17:10:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:44.035322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b63555-ed6b-42c1-b102-e57ed87fd8ce None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:44.136071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b63555-ed6b-42c1-b102-e57ed87fd8ce None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:44.170845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:44.171650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:44.171934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:44.172211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:44.183917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:44.185402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:44.191954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:44.203698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:44.204643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:44.205602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:44.205602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:44.205602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:44.205602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:44.206401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 531/1064] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:44.211668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:44.238317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:44.316134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79f345-9eba-4d07-ba7f-4e14553dbf73 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:44.351081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:44.351752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:44.352032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:44.352305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:44.353036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:44.353377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:44.354747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:44.367782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 534/1065] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:44.373293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:44.478863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe21fae-97dd-4338-933b-d4aa506b4621 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:44.517301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:44.518097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:44.518376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:44.518651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:44.526485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:44.527936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:44.541706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:44.542655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:44.543631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:44.544797 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 532/1066] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:10:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:44.548643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:44.636760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:44.657193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e3b8c-0001-4ff9-abec-53f5b5b1a18d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:44.691910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baedf8-e1cd-492d-aa08-654ee67ab5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:44.692584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baedf8-e1cd-492d-aa08-654ee67ab5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:44.692863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baedf8-e1cd-492d-aa08-654ee67ab5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:44.693134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baedf8-e1cd-492d-aa08-654ee67ab5b6 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:44.694001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baedf8-e1cd-492d-aa08-654ee67ab5b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:44.694304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baedf8-e1cd-492d-aa08-654ee67ab5b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:44.695737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0baedf8-e1cd-492d-aa08-654ee67ab5b6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:44.714442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 535/1067] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:10:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:44.720437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:44.786821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:44.789678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10708ea2-51b2-4194-8712-839157a9768f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:44.790362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10708ea2-51b2-4194-8712-839157a9768f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:44.790651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10708ea2-51b2-4194-8712-839157a9768f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:44.792450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 536/1068] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:44.798753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645edcb4-956e-47b1-9d8c-f69003796346 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:44.799048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645edcb4-956e-47b1-9d8c-f69003796346 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:44.799329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645edcb4-956e-47b1-9d8c-f69003796346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:44.819019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-645edcb4-956e-47b1-9d8c-f69003796346 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:44.820252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-645edcb4-956e-47b1-9d8c-f69003796346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:44.837485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e500b3-bc91-45c8-9f97-b90ddb7b6297 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:44.877759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:44.878843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:44.879242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:44.879771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:44.891519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:44.892817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 537/1069] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:44.893000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:44.910175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:44.911130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:44.912086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:44.912086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:44.912086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:44.912086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:44.912086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 533/1070] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:44.915355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645edcb4-956e-47b1-9d8c-f69003796346 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:44.916660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:44.993866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:45.021435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bf952-bb13-4e03-9e0a-93f9ea6f9539 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.028902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645edcb4-956e-47b1-9d8c-f69003796346 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.031594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:45.057195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.057795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.058080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.058516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.059152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:45.059386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:45.060754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:45.066719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0126a1-9494-4d66-afe1-09c98511f3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.067526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0126a1-9494-4d66-afe1-09c98511f3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.067810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0126a1-9494-4d66-afe1-09c98511f3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.068561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0126a1-9494-4d66-afe1-09c98511f3c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.078150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 534/1071] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:45.079158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:45.085655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0126a1-9494-4d66-afe1-09c98511f3c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5b0126a1-9494-4d66-afe1-09c98511f3c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:45.094430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:45.095429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:45.096298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 538/1072] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:45.101668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0126a1-9494-4d66-afe1-09c98511f3c8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:45.185837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f59579-6388-4339-8a3d-67c8e29e5e1e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.206624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0126a1-9494-4d66-afe1-09c98511f3c8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.224300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.225112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.225441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.225711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.232960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:45.234442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:45.239668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:45.246377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.247050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.247330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.247604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.248354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:45.248448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:45.249551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:45.250901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:45.250901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:45.250901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:45.250901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:45.250901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:45.250901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:45.250901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:45.250901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:45.251450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:45.251543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 535/1073] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:45.253715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:45.255296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:45.263712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 539/1074] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:45.269727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:45.286103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:45.358296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a90e8-06e8-4684-97d4-6639123d5bf1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.375052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19e59b-4e74-423d-b348-250c1398b93a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.393478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.394144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.394452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.394727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.395523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:45.395827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:45.397283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:45.414449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 536/1075] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:45.415624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d8b9e-b04c-4c84-9df4-29ea45a5318a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.415752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:45.416520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8b9e-b04c-4c84-9df4-29ea45a5318a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.416834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8b9e-b04c-4c84-9df4-29ea45a5318a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.417115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d8b9e-b04c-4c84-9df4-29ea45a5318a None admin] PATH_INFO: `/v3/users/673614eb4ff64bd29279b95016e6654f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.418416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d8b9e-b04c-4c84-9df4-29ea45a5318a None admin] RBAC: Authorizing `identity:get_user(user_id=673614eb4ff64bd29279b95016e6654f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:45.419923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d8b9e-b04c-4c84-9df4-29ea45a5318a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:45.425527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 540/1076] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:10:45 2022] GET /identity/v3/users/673614eb4ff64bd29279b95016e6654f => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:45.427315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d8b9e-b04c-4c84-9df4-29ea45a5318a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:45.496984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:45.519499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:45.530830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da45e1d-cb11-4fc1-b674-5a8188b74547 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.540611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d8b9e-b04c-4c84-9df4-29ea45a5318a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.567697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa0527-fb1e-49af-8516-1fef4d55b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.569074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa0527-fb1e-49af-8516-1fef4d55b2e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.569637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa0527-fb1e-49af-8516-1fef4d55b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.569918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa0527-fb1e-49af-8516-1fef4d55b2e3 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/e3241452a88a40da9c9b8fa79c4fc9c0/roles/8a3e8dfadf1142a8adba3fe567890600` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.572138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa0527-fb1e-49af-8516-1fef4d55b2e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=e3241452a88a40da9c9b8fa79c4fc9c0, role_id=8a3e8dfadf1142a8adba3fe567890600)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:45.573614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa0527-fb1e-49af-8516-1fef4d55b2e3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:45.574480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523b4b-bbd4-4f55-a22c-8c586c7f4f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.575362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523b4b-bbd4-4f55-a22c-8c586c7f4f50 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.575643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523b4b-bbd4-4f55-a22c-8c586c7f4f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.575916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e523b4b-bbd4-4f55-a22c-8c586c7f4f50 None admin] PATH_INFO: `/v3/projects/c6b34fe332ce4538b6729042368ac539` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.577109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e523b4b-bbd4-4f55-a22c-8c586c7f4f50 None admin] RBAC: Authorizing `identity:get_project(project_id=c6b34fe332ce4538b6729042368ac539)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:45.578589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e523b4b-bbd4-4f55-a22c-8c586c7f4f50 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:45.581206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 541/1077] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:10:45 2022] GET /identity/v3/projects/c6b34fe332ce4538b6729042368ac539 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:45.587209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523b4b-bbd4-4f55-a22c-8c586c7f4f50 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:45.593487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 537/1078] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:10:45 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/e3241452a88a40da9c9b8fa79c4fc9c0/roles/8a3e8dfadf1142a8adba3fe567890600 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:10:45.702067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e523b4b-bbd4-4f55-a22c-8c586c7f4f50 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:45.733937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:45.743590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be4bcf5-baad-4671-b0f6-da4ce5921e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:45.744509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4bcf5-baad-4671-b0f6-da4ce5921e97 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.744792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4bcf5-baad-4671-b0f6-da4ce5921e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.745067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be4bcf5-baad-4671-b0f6-da4ce5921e97 None admin] PATH_INFO: `/v3/projects/c6b34fe332ce4538b6729042368ac539/users/673614eb4ff64bd29279b95016e6654f/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.747186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4bcf5-baad-4671-b0f6-da4ce5921e97 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6b34fe332ce4538b6729042368ac539, user_id=673614eb4ff64bd29279b95016e6654f, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:45.748647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be4bcf5-baad-4671-b0f6-da4ce5921e97 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:45.764750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 542/1079] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:10:45 2022] PUT /identity/v3/projects/c6b34fe332ce4538b6729042368ac539/users/673614eb4ff64bd29279b95016e6654f/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:10:45.774150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7c541-6bd3-4f34-ae4b-07fd64115abb None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.774479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7c541-6bd3-4f34-ae4b-07fd64115abb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.774720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c7c541-6bd3-4f34-ae4b-07fd64115abb None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.776538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 538/1080] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:45.782453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f1f79-dcab-45c0-a122-d3453ee24f55 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:45.782770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f1f79-dcab-45c0-a122-d3453ee24f55 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:45.783012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684f1f79-dcab-45c0-a122-d3453ee24f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:45.802029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-684f1f79-dcab-45c0-a122-d3453ee24f55 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:45.803560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-684f1f79-dcab-45c0-a122-d3453ee24f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:45.881999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 543/1081] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:45.904140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7c541-6bd3-4f34-ae4b-07fd64115abb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:46.013723 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c7c541-6bd3-4f34-ae4b-07fd64115abb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:46.047247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fd3a7-836e-427a-b9cc-e9a5f1e0720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:46.047898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd3a7-836e-427a-b9cc-e9a5f1e0720d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:46.048179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd3a7-836e-427a-b9cc-e9a5f1e0720d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:46.048449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564fd3a7-836e-427a-b9cc-e9a5f1e0720d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:46.069799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:46.123237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fd3a7-836e-427a-b9cc-e9a5f1e0720d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:46.124694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564fd3a7-836e-427a-b9cc-e9a5f1e0720d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:46.233892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 539/1082] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:46.240518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f1f79-dcab-45c0-a122-d3453ee24f55 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:46.351009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684f1f79-dcab-45c0-a122-d3453ee24f55 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:46.369908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:46.391297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3711bff-5480-40d7-bb12-d1b77a0a343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:46.392106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711bff-5480-40d7-bb12-d1b77a0a343a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:46.392390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711bff-5480-40d7-bb12-d1b77a0a343a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:46.392671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711bff-5480-40d7-bb12-d1b77a0a343a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:46.406222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3711bff-5480-40d7-bb12-d1b77a0a343a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:46.407695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3711bff-5480-40d7-bb12-d1b77a0a343a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e3711bff-5480-40d7-bb12-d1b77a0a343a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:46.426310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:46.427352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:46.428266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:46.428266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:46.428266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:46.428266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:46.428266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 544/1083] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:46.434835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fd3a7-836e-427a-b9cc-e9a5f1e0720d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:46.493268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:46.545395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564fd3a7-836e-427a-b9cc-e9a5f1e0720d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:46.583362 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf392ac-f31b-492c-b081-aaca96c85cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:46.584024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf392ac-f31b-492c-b081-aaca96c85cef None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:46.584300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf392ac-f31b-492c-b081-aaca96c85cef None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:46.584876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf392ac-f31b-492c-b081-aaca96c85cef None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:46.585634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf392ac-f31b-492c-b081-aaca96c85cef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:46.585939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf392ac-f31b-492c-b081-aaca96c85cef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:46.587338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf392ac-f31b-492c-b081-aaca96c85cef None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:46.599498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 540/1084] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:47.730251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6492b1-8d09-4c9f-94e3-fddc7df41789 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:47.730783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6492b1-8d09-4c9f-94e3-fddc7df41789 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:47.730783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6492b1-8d09-4c9f-94e3-fddc7df41789 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:47.732198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 545/1085] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:47.737784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585df63-46cf-4efc-bedb-19b5de714a39 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:47.738235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585df63-46cf-4efc-bedb-19b5de714a39 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:47.738399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a585df63-46cf-4efc-bedb-19b5de714a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:47.756103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a585df63-46cf-4efc-bedb-19b5de714a39 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:47.757224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a585df63-46cf-4efc-bedb-19b5de714a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:47.830263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 541/1086] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:47.853983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6492b1-8d09-4c9f-94e3-fddc7df41789 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:47.880688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21331433-79f2-46fa-b933-bb6a0b925389 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:47.881059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21331433-79f2-46fa-b933-bb6a0b925389 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:47.881383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21331433-79f2-46fa-b933-bb6a0b925389 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:47.882744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 542/1087] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:47.887728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421d1f3-590b-425a-bd26-b883799f9634 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:47.888014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421d1f3-590b-425a-bd26-b883799f9634 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:47.888293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421d1f3-590b-425a-bd26-b883799f9634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:47.904892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9421d1f3-590b-425a-bd26-b883799f9634 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:47.906136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9421d1f3-590b-425a-bd26-b883799f9634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:47.941334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:47.960751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6492b1-8d09-4c9f-94e3-fddc7df41789 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:47.975997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 543/1088] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:10:47.993926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334a27d-898c-4721-b4e3-316e56fc37fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:47.994739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a27d-898c-4721-b4e3-316e56fc37fd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:47.995025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a27d-898c-4721-b4e3-316e56fc37fd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:47.995298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334a27d-898c-4721-b4e3-316e56fc37fd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:47.999247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421d1f3-590b-425a-bd26-b883799f9634 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.005622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334a27d-898c-4721-b4e3-316e56fc37fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0334a27d-898c-4721-b4e3-316e56fc37fd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:48.013826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:48.014782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:48.015704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 546/1089] 10.222.0.66 () {56 vars in 1120 bytes} [Thu Jun 16 17:10:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:48.020829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334a27d-898c-4721-b4e3-316e56fc37fd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.051016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.096795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421d1f3-590b-425a-bd26-b883799f9634 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.121716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334a27d-898c-4721-b4e3-316e56fc37fd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.129911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22354623-262d-4f23-b1e1-334e68463874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.130703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22354623-262d-4f23-b1e1-334e68463874 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.130980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22354623-262d-4f23-b1e1-334e68463874 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.131253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22354623-262d-4f23-b1e1-334e68463874 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.141086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22354623-262d-4f23-b1e1-334e68463874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-22354623-262d-4f23-b1e1-334e68463874 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:10:48.148654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:48.149740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:48.150616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 544/1090] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:10:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:48.155138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.155804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.155936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22354623-262d-4f23-b1e1-334e68463874 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.156058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.156338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.157088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:48.157443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.158810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.167332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 547/1091] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:48.173102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.187252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.260938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22354623-262d-4f23-b1e1-334e68463874 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.270892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.279681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2b11-1876-4681-a229-e7b1f4f2dc62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.296210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.296876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.297156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.297461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.298205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:48.298508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.299866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.308180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 545/1092] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:48.313936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.314738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.315541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.315820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.316091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.327234 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.328803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:48.347445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:48.348611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:48.349825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:48.349825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:48.349825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:48.349825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:48.349825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 548/1093] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:10:48.354679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.416891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.426610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66093fe3-74ff-4850-8304-eb8729043e4a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.457893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653a026-28ba-45d4-a2ba-21164dab3432 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.463663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a17fc-7443-4890-bd3c-88950610d486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.464578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a17fc-7443-4890-bd3c-88950610d486 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.464845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a17fc-7443-4890-bd3c-88950610d486 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.465151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a17fc-7443-4890-bd3c-88950610d486 None admin] PATH_INFO: `/v3/users/673614eb4ff64bd29279b95016e6654f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.466428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a17fc-7443-4890-bd3c-88950610d486 None admin] RBAC: Authorizing `identity:get_user(user_id=673614eb4ff64bd29279b95016e6654f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.467877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a17fc-7443-4890-bd3c-88950610d486 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.473617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 546/1094] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/users/673614eb4ff64bd29279b95016e6654f => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:48.476112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a17fc-7443-4890-bd3c-88950610d486 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.497914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.498583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.498863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.499142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.499859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:48.500163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.501552 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.513499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 549/1095] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:48.519151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.522839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.576300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a17fc-7443-4890-bd3c-88950610d486 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.608963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec5f3-dc4f-4ab5-9479-9a7c544cdcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.609632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec5f3-dc4f-4ab5-9479-9a7c544cdcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.609912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec5f3-dc4f-4ab5-9479-9a7c544cdcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.610189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ec5f3-dc4f-4ab5-9479-9a7c544cdcf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.616555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a7c1f-3900-42d5-a8a1-22273190674f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.639596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.651335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.652130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.652405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.652669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.659787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.661336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:10:48.675393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:48.676994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:10:48.678034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 550/1096] 10.222.0.66 () {56 vars in 1093 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:48.679950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec5f3-dc4f-4ab5-9479-9a7c544cdcf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.681694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ec5f3-dc4f-4ab5-9479-9a7c544cdcf3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.682537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.744959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.781034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 547/1097] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 17:10:48.784533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee366625-1849-46e6-8288-059cfda6370b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.786820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec5f3-dc4f-4ab5-9479-9a7c544cdcf3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.820975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.821672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.821951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.822221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.823053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:48.823356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.824781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.842427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 551/1098] 10.222.0.66 () {56 vars in 1133 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:48.845268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.848595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.886984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ec5f3-dc4f-4ab5-9479-9a7c544cdcf3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.922295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713918-f9c1-45d1-97c3-c7eae929145c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.923155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713918-f9c1-45d1-97c3-c7eae929145c None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.923436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713918-f9c1-45d1-97c3-c7eae929145c None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.923719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f713918-f9c1-45d1-97c3-c7eae929145c None admin] PATH_INFO: `/v3/projects/c6b34fe332ce4538b6729042368ac539` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:48.924917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713918-f9c1-45d1-97c3-c7eae929145c None admin] RBAC: Authorizing `identity:get_project(project_id=c6b34fe332ce4538b6729042368ac539)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:48.926415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f713918-f9c1-45d1-97c3-c7eae929145c None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:48.928137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 548/1099] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/projects/c6b34fe332ce4538b6729042368ac539 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:48.934123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713918-f9c1-45d1-97c3-c7eae929145c None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:48.948415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c5e0e-fc46-4b13-b85d-83e313a8d340 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:48.972037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:48.985683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d7b7-b357-4713-9a82-8be0f9ff25da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:48.986355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d7b7-b357-4713-9a82-8be0f9ff25da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:48.986622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d7b7-b357-4713-9a82-8be0f9ff25da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:48.986899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895d7b7-b357-4713-9a82-8be0f9ff25da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:49.032641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f713918-f9c1-45d1-97c3-c7eae929145c None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:49.053491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d7b7-b357-4713-9a82-8be0f9ff25da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:49.054926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895d7b7-b357-4713-9a82-8be0f9ff25da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:49.057829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:49.065862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:49.066718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:49.066995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:49.067273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:49.068091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=c6b34fe332ce4538b6729042368ac539, user.id=673614eb4ff64bd29279b95016e6654f) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:49.068400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:49.069844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:49.075629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 549/1100] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/role_assignments?user.id=673614eb4ff64bd29279b95016e6654f&scope.project.id=c6b34fe332ce4538b6729042368ac539&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:10:49.160925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 552/1101] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:49.167231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:49.276627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e69f6-3dab-4c78-8c68-0191d4468733 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:49.313253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:49.314015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:49.314292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:49.314568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:49.325571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:49.327015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:10:49.344444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:49.345440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:49.346360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:10:49.346360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:10:49.346360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:10:49.346360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:49.349347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 550/1102] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:49.351440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d7b7-b357-4713-9a82-8be0f9ff25da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:49.376004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:49.453600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895d7b7-b357-4713-9a82-8be0f9ff25da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:49.490912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:49.491376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:49.491654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:49.491927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:49.492654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:49.492957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:49.494369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:49.506436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 553/1103] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:49.512160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:49.557468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:49.610902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae78d69-2ecb-4333-8aa8-53384c6a9978 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:49.645035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:49.645857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:49.646137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:49.646392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:49.653021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:49.654485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:10:49.668729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:10:49.669731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:10:49.670732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 551/1104] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:10:49.675932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:49.759650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:49.778781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f138c-0e70-4f91-bfc2-f5eca1c71ea5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:49.811638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b46f8-1ad8-4d10-ae0a-3c40461bbaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:49.812298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b46f8-1ad8-4d10-ae0a-3c40461bbaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:49.812578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b46f8-1ad8-4d10-ae0a-3c40461bbaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:49.812853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b46f8-1ad8-4d10-ae0a-3c40461bbaa5 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:49.813670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b46f8-1ad8-4d10-ae0a-3c40461bbaa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:49.813978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b46f8-1ad8-4d10-ae0a-3c40461bbaa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:49.815450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b46f8-1ad8-4d10-ae0a-3c40461bbaa5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:49.827547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 554/1105] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:10:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:49.833708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:49.942072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429266a-3b2b-462a-b26c-a12831ccb56d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:49.944569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:49.978248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e5f914-15db-4e5a-8056-47e1ac1ca90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:49.978891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f914-15db-4e5a-8056-47e1ac1ca90d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:49.979174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f914-15db-4e5a-8056-47e1ac1ca90d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:49.979443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e5f914-15db-4e5a-8056-47e1ac1ca90d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:49.980255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f914-15db-4e5a-8056-47e1ac1ca90d None admin] RBAC: Adding query filter params (role.id=8a3e8dfadf1142a8adba3fe567890600, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=e3241452a88a40da9c9b8fa79c4fc9c0) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:49.980564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f914-15db-4e5a-8056-47e1ac1ca90d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:49.982016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e5f914-15db-4e5a-8056-47e1ac1ca90d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:49.997218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 552/1106] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:49 2022] GET /identity/v3/role_assignments?user.id=e3241452a88a40da9c9b8fa79c4fc9c0&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=8a3e8dfadf1142a8adba3fe567890600 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:51.200004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effcb8-2b0a-4531-aeca-d9a8ee4aab12 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:51.200597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effcb8-2b0a-4531-aeca-d9a8ee4aab12 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:51.200597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13effcb8-2b0a-4531-aeca-d9a8ee4aab12 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:51.201963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 555/1107] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:51.208572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540946-4853-4f23-9e91-462612935e18 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:51.209312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540946-4853-4f23-9e91-462612935e18 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:51.209807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540946-4853-4f23-9e91-462612935e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:51.231984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e6540946-4853-4f23-9e91-462612935e18 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:51.233165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e6540946-4853-4f23-9e91-462612935e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:51.304862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 553/1108] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:51.330425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13effcb8-2b0a-4531-aeca-d9a8ee4aab12 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:51.449577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13effcb8-2b0a-4531-aeca-d9a8ee4aab12 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:51.488341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56682b4f-9256-4b9f-bf12-a964c5c66dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:51.489320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56682b4f-9256-4b9f-bf12-a964c5c66dad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:51.489604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56682b4f-9256-4b9f-bf12-a964c5c66dad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:51.489880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56682b4f-9256-4b9f-bf12-a964c5c66dad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:51.491130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56682b4f-9256-4b9f-bf12-a964c5c66dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:51.492576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56682b4f-9256-4b9f-bf12-a964c5c66dad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:51.494262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 556/1109] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:10:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:51.503208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6540946-4853-4f23-9e91-462612935e18 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:51.606002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:51.609636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6540946-4853-4f23-9e91-462612935e18 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:51.647108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaedc59-02b6-466c-9067-d3a4ffbafc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:51.647760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaedc59-02b6-466c-9067-d3a4ffbafc66 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:51.648039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaedc59-02b6-466c-9067-d3a4ffbafc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:51.648313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaedc59-02b6-466c-9067-d3a4ffbafc66 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:51.649013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaedc59-02b6-466c-9067-d3a4ffbafc66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:51.650475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaedc59-02b6-466c-9067-d3a4ffbafc66 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:51.672592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 554/1110] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:10:51 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:10:53.774555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93868217-ac69-4a52-a0fc-1ae532a0e204 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:53.775092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93868217-ac69-4a52-a0fc-1ae532a0e204 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:53.775092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93868217-ac69-4a52-a0fc-1ae532a0e204 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:53.777079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 557/1111] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:53.785820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e66b-c7a2-4300-b963-5a32fbb6769f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:53.786220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e66b-c7a2-4300-b963-5a32fbb6769f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:53.786318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e66b-c7a2-4300-b963-5a32fbb6769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:53.809627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c646e66b-c7a2-4300-b963-5a32fbb6769f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:53.810854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c646e66b-c7a2-4300-b963-5a32fbb6769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:53.884645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 555/1112] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:53.912798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93868217-ac69-4a52-a0fc-1ae532a0e204 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:53.927687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:54.022945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93868217-ac69-4a52-a0fc-1ae532a0e204 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:54.059956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04965553-23b2-4aed-89bd-f6e875665644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:54.060748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965553-23b2-4aed-89bd-f6e875665644 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:54.061026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965553-23b2-4aed-89bd-f6e875665644 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:54.061329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965553-23b2-4aed-89bd-f6e875665644 None admin] PATH_INFO: `/v3/roles/774686ef0c4c42ba97f56e75b67d313e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:54.062858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04965553-23b2-4aed-89bd-f6e875665644 None admin] RBAC: Authorizing `identity:get_role(role_id=774686ef0c4c42ba97f56e75b67d313e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:54.064254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04965553-23b2-4aed-89bd-f6e875665644 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:54.065643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 558/1113] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:10:53 2022] GET /identity/v3/roles/774686ef0c4c42ba97f56e75b67d313e => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:54.072096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646e66b-c7a2-4300-b963-5a32fbb6769f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:54.157203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:54.174078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646e66b-c7a2-4300-b963-5a32fbb6769f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:54.208568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25126aa-4ecc-4cf2-b4b5-3658efca7f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:54.209835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25126aa-4ecc-4cf2-b4b5-3658efca7f00 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:54.210116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25126aa-4ecc-4cf2-b4b5-3658efca7f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:54.210389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25126aa-4ecc-4cf2-b4b5-3658efca7f00 None admin] PATH_INFO: `/v3/users/c07902c5335246348e80005c0a271332` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:54.223471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25126aa-4ecc-4cf2-b4b5-3658efca7f00 None admin] RBAC: Authorizing `identity:get_user(user_id=c07902c5335246348e80005c0a271332)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:54.224907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25126aa-4ecc-4cf2-b4b5-3658efca7f00 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:54.226597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 556/1114] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:10:54 2022] GET /identity/v3/users/c07902c5335246348e80005c0a271332 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:54.232690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04965553-23b2-4aed-89bd-f6e875665644 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:54.319569 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:54.343361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04965553-23b2-4aed-89bd-f6e875665644 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:54.379574 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57fe24-0f55-4928-ae82-106037e1d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:54.380222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57fe24-0f55-4928-ae82-106037e1d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:54.380507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57fe24-0f55-4928-ae82-106037e1d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:54.380794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57fe24-0f55-4928-ae82-106037e1d0e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:54.451506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57fe24-0f55-4928-ae82-106037e1d0e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:54.452955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57fe24-0f55-4928-ae82-106037e1d0e2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:54.543046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:54.566654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 559/1115] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:10:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:10:54.576043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25126aa-4ecc-4cf2-b4b5-3658efca7f00 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:54.684060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25126aa-4ecc-4cf2-b4b5-3658efca7f00 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:54.719582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dcd671-7255-4faa-b3f9-15ee2a4dbecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:54.720438 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcd671-7255-4faa-b3f9-15ee2a4dbecd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:54.720717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcd671-7255-4faa-b3f9-15ee2a4dbecd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:54.720986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dcd671-7255-4faa-b3f9-15ee2a4dbecd None admin] PATH_INFO: `/v3/projects/c6b34fe332ce4538b6729042368ac539` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:54.722235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dcd671-7255-4faa-b3f9-15ee2a4dbecd None admin] RBAC: Authorizing `identity:get_project(project_id=c6b34fe332ce4538b6729042368ac539)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:54.723703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dcd671-7255-4faa-b3f9-15ee2a4dbecd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:54.725467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 557/1116] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:10:54 2022] GET /identity/v3/projects/c6b34fe332ce4538b6729042368ac539 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:54.732725 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57fe24-0f55-4928-ae82-106037e1d0e2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:54.838368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57fe24-0f55-4928-ae82-106037e1d0e2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:54.872939 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf3fd1-f997-41c1-a8ee-74e3c81ae7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:54.873613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf3fd1-f997-41c1-a8ee-74e3c81ae7ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:54.873892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf3fd1-f997-41c1-a8ee-74e3c81ae7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:54.874168 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf3fd1-f997-41c1-a8ee-74e3c81ae7ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:54.874996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf3fd1-f997-41c1-a8ee-74e3c81ae7ed None admin] RBAC: Adding query filter params (role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=c6b34fe332ce4538b6729042368ac539, user.id=c07902c5335246348e80005c0a271332) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:10:54.875306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf3fd1-f997-41c1-a8ee-74e3c81ae7ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:54.876718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bf3fd1-f997-41c1-a8ee-74e3c81ae7ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:54.882444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 560/1117] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:10:54 2022] GET /identity/v3/role_assignments?user.id=c07902c5335246348e80005c0a271332&scope.project.id=c6b34fe332ce4538b6729042368ac539&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:56.947509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f14ee-08c3-44a5-9c36-2ac671805c8c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:56.948335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f14ee-08c3-44a5-9c36-2ac671805c8c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:56.948335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f14ee-08c3-44a5-9c36-2ac671805c8c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:56.950578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 558/1118] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:56.956262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7be2a0-460c-4533-b210-ad1f54ea6767 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:56.956798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7be2a0-460c-4533-b210-ad1f54ea6767 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:56.957010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7be2a0-460c-4533-b210-ad1f54ea6767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:56.975371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5c7be2a0-460c-4533-b210-ad1f54ea6767 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:56.976545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5c7be2a0-460c-4533-b210-ad1f54ea6767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:57.012772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:57.047537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 561/1119] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:57.070861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f14ee-08c3-44a5-9c36-2ac671805c8c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:57.084135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:57.185755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f14ee-08c3-44a5-9c36-2ac671805c8c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:57.223666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d83f34-cc2f-408d-acae-49d4ee58f328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:57.224458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d83f34-cc2f-408d-acae-49d4ee58f328 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:57.224735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d83f34-cc2f-408d-acae-49d4ee58f328 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:57.225040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d83f34-cc2f-408d-acae-49d4ee58f328 None admin] PATH_INFO: `/v3/roles/774686ef0c4c42ba97f56e75b67d313e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:57.226707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d83f34-cc2f-408d-acae-49d4ee58f328 None admin] RBAC: Authorizing `identity:get_role(role_id=774686ef0c4c42ba97f56e75b67d313e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:57.228093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d83f34-cc2f-408d-acae-49d4ee58f328 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:57.229538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 559/1120] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:10:57 2022] GET /identity/v3/roles/774686ef0c4c42ba97f56e75b67d313e => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:57.235854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7be2a0-460c-4533-b210-ad1f54ea6767 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:57.344500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7be2a0-460c-4533-b210-ad1f54ea6767 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:57.380807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5e86-5e7e-4b35-861b-ca7888fb3096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:57.381708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5e86-5e7e-4b35-861b-ca7888fb3096 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:57.381987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5e86-5e7e-4b35-861b-ca7888fb3096 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:57.382264 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5e86-5e7e-4b35-861b-ca7888fb3096 None admin] PATH_INFO: `/v3/users/c07902c5335246348e80005c0a271332` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:57.383480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d5e86-5e7e-4b35-861b-ca7888fb3096 None admin] RBAC: Authorizing `identity:get_user(user_id=c07902c5335246348e80005c0a271332)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:57.384915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d5e86-5e7e-4b35-861b-ca7888fb3096 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:57.386586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 562/1121] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:10:57 2022] GET /identity/v3/users/c07902c5335246348e80005c0a271332 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:57.392781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d83f34-cc2f-408d-acae-49d4ee58f328 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:57.473278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:57.508186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d83f34-cc2f-408d-acae-49d4ee58f328 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:57.545590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9619eb6-e764-4121-8e8e-54bc62ad58f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:57.546459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9619eb6-e764-4121-8e8e-54bc62ad58f4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:57.546742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9619eb6-e764-4121-8e8e-54bc62ad58f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:57.547025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9619eb6-e764-4121-8e8e-54bc62ad58f4 None admin] PATH_INFO: `/v3/projects/c6b34fe332ce4538b6729042368ac539` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:57.548270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9619eb6-e764-4121-8e8e-54bc62ad58f4 None admin] RBAC: Authorizing `identity:get_project(project_id=c6b34fe332ce4538b6729042368ac539)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:57.549749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9619eb6-e764-4121-8e8e-54bc62ad58f4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:57.551998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 560/1122] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:10:57 2022] GET /identity/v3/projects/c6b34fe332ce4538b6729042368ac539 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:10:57.558580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5e86-5e7e-4b35-861b-ca7888fb3096 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:10:57.601758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:10:57.662902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5e86-5e7e-4b35-861b-ca7888fb3096 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:10:57.697920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20197d40-3afa-4009-97bf-807772368370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:10:57.698804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20197d40-3afa-4009-97bf-807772368370 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:57.699081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20197d40-3afa-4009-97bf-807772368370 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:57.699353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20197d40-3afa-4009-97bf-807772368370 None admin] PATH_INFO: `/v3/projects/c6b34fe332ce4538b6729042368ac539/users/c07902c5335246348e80005c0a271332/roles/774686ef0c4c42ba97f56e75b67d313e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:57.701670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20197d40-3afa-4009-97bf-807772368370 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6b34fe332ce4538b6729042368ac539, user_id=c07902c5335246348e80005c0a271332, role_id=774686ef0c4c42ba97f56e75b67d313e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:10:57.703109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20197d40-3afa-4009-97bf-807772368370 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:10:57.718683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 563/1123] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:10:57 2022] PUT /identity/v3/projects/c6b34fe332ce4538b6729042368ac539/users/c07902c5335246348e80005c0a271332/roles/774686ef0c4c42ba97f56e75b67d313e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:10:59.782337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029770d0-2af1-4d03-a394-b00fdad2a584 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:59.783139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029770d0-2af1-4d03-a394-b00fdad2a584 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:59.783139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029770d0-2af1-4d03-a394-b00fdad2a584 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:59.785696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 561/1124] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:10:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:10:59.794404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f79da-9bb1-429e-909f-8b9bad0ab1e9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:10:59.795159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f79da-9bb1-429e-909f-8b9bad0ab1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:10:59.795290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94f79da-9bb1-429e-909f-8b9bad0ab1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:10:59.814679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a94f79da-9bb1-429e-909f-8b9bad0ab1e9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:10:59.815892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a94f79da-9bb1-429e-909f-8b9bad0ab1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:10:59.891070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 564/1125] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:10:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:10:59.915645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029770d0-2af1-4d03-a394-b00fdad2a584 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:00.029763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029770d0-2af1-4d03-a394-b00fdad2a584 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:00.068378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea9798-f513-4507-9bcb-47f05dcc979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:00.069170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea9798-f513-4507-9bcb-47f05dcc979d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:00.069477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea9798-f513-4507-9bcb-47f05dcc979d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:00.069754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea9798-f513-4507-9bcb-47f05dcc979d None admin] PATH_INFO: `/v3/roles/774686ef0c4c42ba97f56e75b67d313e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:00.071286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea9798-f513-4507-9bcb-47f05dcc979d None admin] RBAC: Authorizing `identity:get_role(role_id=774686ef0c4c42ba97f56e75b67d313e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:00.072651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feea9798-f513-4507-9bcb-47f05dcc979d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:00.074034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 562/1126] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:10:59 2022] GET /identity/v3/roles/774686ef0c4c42ba97f56e75b67d313e => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:00.080661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f79da-9bb1-429e-909f-8b9bad0ab1e9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:00.124928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:00.197567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94f79da-9bb1-429e-909f-8b9bad0ab1e9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:00.233386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b61f4-405a-41f3-9ce5-a0ae10bc5b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:00.234295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b61f4-405a-41f3-9ce5-a0ae10bc5b0b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:00.234571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b61f4-405a-41f3-9ce5-a0ae10bc5b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:00.234843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b61f4-405a-41f3-9ce5-a0ae10bc5b0b None admin] PATH_INFO: `/v3/users/c07902c5335246348e80005c0a271332` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:00.236181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b61f4-405a-41f3-9ce5-a0ae10bc5b0b None admin] RBAC: Authorizing `identity:get_user(user_id=c07902c5335246348e80005c0a271332)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:00.237648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b61f4-405a-41f3-9ce5-a0ae10bc5b0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:00.239319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 565/1127] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:11:00 2022] GET /identity/v3/users/c07902c5335246348e80005c0a271332 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:00.245867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea9798-f513-4507-9bcb-47f05dcc979d None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:00.265239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:00.354743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea9798-f513-4507-9bcb-47f05dcc979d None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:00.392850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59132c-6a5b-494a-b34a-44fdf7e94cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:00.393533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59132c-6a5b-494a-b34a-44fdf7e94cdd None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:00.393811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59132c-6a5b-494a-b34a-44fdf7e94cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:00.394125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce59132c-6a5b-494a-b34a-44fdf7e94cdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:00.463781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59132c-6a5b-494a-b34a-44fdf7e94cdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:00.465225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce59132c-6a5b-494a-b34a-44fdf7e94cdd None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:00.479659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:00.579916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 563/1128] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:11:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:11:00.587592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b61f4-405a-41f3-9ce5-a0ae10bc5b0b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:00.699064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:00.703021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b61f4-405a-41f3-9ce5-a0ae10bc5b0b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:00.739024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bb7f4-0bb9-4d80-8b36-67f91ff26250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:00.739880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb7f4-0bb9-4d80-8b36-67f91ff26250 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:00.740156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb7f4-0bb9-4d80-8b36-67f91ff26250 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:00.740423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bb7f4-0bb9-4d80-8b36-67f91ff26250 None admin] PATH_INFO: `/v3/projects/c6b34fe332ce4538b6729042368ac539` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:00.741757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bb7f4-0bb9-4d80-8b36-67f91ff26250 None admin] RBAC: Authorizing `identity:get_project(project_id=c6b34fe332ce4538b6729042368ac539)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:00.743186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bb7f4-0bb9-4d80-8b36-67f91ff26250 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:00.744900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 566/1129] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:11:00 2022] GET /identity/v3/projects/c6b34fe332ce4538b6729042368ac539 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:00.752154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59132c-6a5b-494a-b34a-44fdf7e94cdd None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:00.862764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce59132c-6a5b-494a-b34a-44fdf7e94cdd None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:00.876397 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:00.911192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f03ea1a-c13a-462d-9b12-fa2b03b27eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:00.911845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03ea1a-c13a-462d-9b12-fa2b03b27eea None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:00.912129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03ea1a-c13a-462d-9b12-fa2b03b27eea None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:00.912408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f03ea1a-c13a-462d-9b12-fa2b03b27eea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:00.913230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03ea1a-c13a-462d-9b12-fa2b03b27eea None admin] RBAC: Adding query filter params (role.id=774686ef0c4c42ba97f56e75b67d313e, scope.project.id=c6b34fe332ce4538b6729042368ac539, user.id=c07902c5335246348e80005c0a271332) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:00.913573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03ea1a-c13a-462d-9b12-fa2b03b27eea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:00.915003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f03ea1a-c13a-462d-9b12-fa2b03b27eea None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:00.921051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 564/1130] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:11:00 2022] GET /identity/v3/role_assignments?user.id=c07902c5335246348e80005c0a271332&scope.project.id=c6b34fe332ce4538b6729042368ac539&role.id=774686ef0c4c42ba97f56e75b67d313e => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:03.094294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36699831-c116-4780-bc61-40fbb8063c93 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:03.094878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36699831-c116-4780-bc61-40fbb8063c93 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:03.094878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36699831-c116-4780-bc61-40fbb8063c93 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:03.096993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 567/1131] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:03.105308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7442ec8-9dba-4154-8328-72f04da246c9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:03.105751 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7442ec8-9dba-4154-8328-72f04da246c9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:03.105858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7442ec8-9dba-4154-8328-72f04da246c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:03.124239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c7442ec8-9dba-4154-8328-72f04da246c9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:03.125471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c7442ec8-9dba-4154-8328-72f04da246c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:03.197439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 565/1132] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:11:03.230975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36699831-c116-4780-bc61-40fbb8063c93 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:03.336880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36699831-c116-4780-bc61-40fbb8063c93 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:03.372797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0162c-2bf5-41c8-af52-66d2c3b8285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:03.373536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0162c-2bf5-41c8-af52-66d2c3b8285b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:03.373822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0162c-2bf5-41c8-af52-66d2c3b8285b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:03.374098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0162c-2bf5-41c8-af52-66d2c3b8285b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:03.375892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0162c-2bf5-41c8-af52-66d2c3b8285b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:03.378180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d0162c-2bf5-41c8-af52-66d2c3b8285b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:03.379924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 568/1133] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:11:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:03.386321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7442ec8-9dba-4154-8328-72f04da246c9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:03.489758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7442ec8-9dba-4154-8328-72f04da246c9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:03.499224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:03.525220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0b2ff0-b1f1-4a0c-b8b2-e03ff7c66604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:03.525905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b2ff0-b1f1-4a0c-b8b2-e03ff7c66604 None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:03.526192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b2ff0-b1f1-4a0c-b8b2-e03ff7c66604 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:03.526471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0b2ff0-b1f1-4a0c-b8b2-e03ff7c66604 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:03.527179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b2ff0-b1f1-4a0c-b8b2-e03ff7c66604 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:03.528609 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0b2ff0-b1f1-4a0c-b8b2-e03ff7c66604 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:03.556087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 566/1134] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:11:03 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:11:05.694122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ceb2e-0300-4564-86e4-39305c03de2e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:05.695004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ceb2e-0300-4564-86e4-39305c03de2e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:05.695004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ceb2e-0300-4564-86e4-39305c03de2e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:05.697148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 569/1135] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:05.702722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced0a88-e177-4922-91a1-781811f7f05b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:05.703186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced0a88-e177-4922-91a1-781811f7f05b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:05.703286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced0a88-e177-4922-91a1-781811f7f05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:05.722013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-dced0a88-e177-4922-91a1-781811f7f05b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:05.723230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dced0a88-e177-4922-91a1-781811f7f05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:05.792079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 567/1136] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:05.816582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ceb2e-0300-4564-86e4-39305c03de2e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:05.857387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:05.931853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ceb2e-0300-4564-86e4-39305c03de2e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:05.966626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e4903-7638-4e93-ab8f-be7fcff9cc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:05.967409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e4903-7638-4e93-ab8f-be7fcff9cc25 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:05.967695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e4903-7638-4e93-ab8f-be7fcff9cc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:05.967948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e4903-7638-4e93-ab8f-be7fcff9cc25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:05.969128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e4903-7638-4e93-ab8f-be7fcff9cc25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:05.970591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e4903-7638-4e93-ab8f-be7fcff9cc25 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:05.972240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 570/1137] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:11:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:05.980742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced0a88-e177-4922-91a1-781811f7f05b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:06.074471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:06.085376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced0a88-e177-4922-91a1-781811f7f05b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:06.120601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f375f4e-906a-440c-9a4b-5f5f655169aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:06.121292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375f4e-906a-440c-9a4b-5f5f655169aa None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:06.121575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375f4e-906a-440c-9a4b-5f5f655169aa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:06.121856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f375f4e-906a-440c-9a4b-5f5f655169aa None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:06.122557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f375f4e-906a-440c-9a4b-5f5f655169aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:06.123982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f375f4e-906a-440c-9a4b-5f5f655169aa None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:06.146898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 568/1138] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:11:05 2022] POST /identity/v3/users => generated 286 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:11:08.261613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231504e-9f66-48e6-a7c5-0afc84cf400c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:08.262551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231504e-9f66-48e6-a7c5-0afc84cf400c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:08.262551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231504e-9f66-48e6-a7c5-0afc84cf400c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:08.264666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 571/1139] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:08.271953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf087-0433-4332-bbe5-b17a8e2907aa None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:08.272387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf087-0433-4332-bbe5-b17a8e2907aa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:08.272501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf087-0433-4332-bbe5-b17a8e2907aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:08.290674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-902cf087-0433-4332-bbe5-b17a8e2907aa None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:08.291762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-902cf087-0433-4332-bbe5-b17a8e2907aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:08.361188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 569/1140] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:11:08.391329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231504e-9f66-48e6-a7c5-0afc84cf400c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:08.508661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231504e-9f66-48e6-a7c5-0afc84cf400c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:08.510237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:08.555633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379d567-1bc7-451d-97ed-eac16e830895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:08.556358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d567-1bc7-451d-97ed-eac16e830895 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:08.556649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d567-1bc7-451d-97ed-eac16e830895 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:08.556927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379d567-1bc7-451d-97ed-eac16e830895 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:08.568500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e379d567-1bc7-451d-97ed-eac16e830895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e379d567-1bc7-451d-97ed-eac16e830895 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:11:08.576139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:11:08.577364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:11:08.578263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 572/1141] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:11:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:11:08.584147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cf087-0433-4332-bbe5-b17a8e2907aa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:08.666900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:08.699687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cf087-0433-4332-bbe5-b17a8e2907aa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:08.735095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:08.735759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:08.736038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:08.736309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:08.737042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:08.737394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:08.738846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:08.746769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 570/1142] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:11:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:08.752782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379d567-1bc7-451d-97ed-eac16e830895 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:08.863122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e379d567-1bc7-451d-97ed-eac16e830895 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:08.896827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6b176-ce27-44a4-a2c0-c72a3ec05fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:08.897714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6b176-ce27-44a4-a2c0-c72a3ec05fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:08.897987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6b176-ce27-44a4-a2c0-c72a3ec05fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:08.898253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b6b176-ce27-44a4-a2c0-c72a3ec05fc9 None admin] PATH_INFO: `/v3/users/8777022683b5416491d4fbd5fe27d3e7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:08.912283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6b176-ce27-44a4-a2c0-c72a3ec05fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=8777022683b5416491d4fbd5fe27d3e7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:08.913750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b6b176-ce27-44a4-a2c0-c72a3ec05fc9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:08.915386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 573/1143] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:11:08 2022] GET /identity/v3/users/8777022683b5416491d4fbd5fe27d3e7 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:08.921989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:09.027456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af14c7-c947-4ba4-a621-f49fb00f56ec None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:09.061112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c1f6-d3c2-42e6-ba4b-9dc941d2d31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:09.061772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c1f6-d3c2-42e6-ba4b-9dc941d2d31a None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:09.062050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c1f6-d3c2-42e6-ba4b-9dc941d2d31a None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:09.062320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c1f6-d3c2-42e6-ba4b-9dc941d2d31a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:09.070909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:09.130991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30c1f6-d3c2-42e6-ba4b-9dc941d2d31a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:09.132351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30c1f6-d3c2-42e6-ba4b-9dc941d2d31a None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:09.231201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 571/1144] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:11:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:11:09.237213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6b176-ce27-44a4-a2c0-c72a3ec05fc9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:09.259567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:09.341079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b6b176-ce27-44a4-a2c0-c72a3ec05fc9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:09.379054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adba022-0d6b-448e-9f44-fe2685ad8371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:09.379914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adba022-0d6b-448e-9f44-fe2685ad8371 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:09.380194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adba022-0d6b-448e-9f44-fe2685ad8371 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:09.380454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adba022-0d6b-448e-9f44-fe2685ad8371 None admin] PATH_INFO: `/v3/projects/ed7610999761466eb9edccfe7cc0577a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:09.381823 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adba022-0d6b-448e-9f44-fe2685ad8371 None admin] RBAC: Authorizing `identity:get_project(project_id=ed7610999761466eb9edccfe7cc0577a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:09.383199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adba022-0d6b-448e-9f44-fe2685ad8371 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:09.384875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 574/1145] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:11:09 2022] GET /identity/v3/projects/ed7610999761466eb9edccfe7cc0577a => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:09.391161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c1f6-d3c2-42e6-ba4b-9dc941d2d31a None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:09.436274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:09.508545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30c1f6-d3c2-42e6-ba4b-9dc941d2d31a None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:09.557390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be3e2d-50a6-468c-89a4-ccc065bc5ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:09.558042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be3e2d-50a6-468c-89a4-ccc065bc5ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:09.558322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be3e2d-50a6-468c-89a4-ccc065bc5ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:09.558593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be3e2d-50a6-468c-89a4-ccc065bc5ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:09.559415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be3e2d-50a6-468c-89a4-ccc065bc5ea2 None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=ed7610999761466eb9edccfe7cc0577a, user.id=8777022683b5416491d4fbd5fe27d3e7) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:09.559730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be3e2d-50a6-468c-89a4-ccc065bc5ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:09.561160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be3e2d-50a6-468c-89a4-ccc065bc5ea2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:09.566825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 572/1146] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:11:09 2022] GET /identity/v3/role_assignments?user.id=8777022683b5416491d4fbd5fe27d3e7&scope.project.id=ed7610999761466eb9edccfe7cc0577a&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:11.672175 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ac89-9d48-4d9a-9748-99c96df5b1a0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:11.672727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ac89-9d48-4d9a-9748-99c96df5b1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:11.672727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355ac89-9d48-4d9a-9748-99c96df5b1a0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:11.674139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 575/1147] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:11.681022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c57752-7eb3-4d64-a109-28c9c769fc4a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:11.681772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c57752-7eb3-4d64-a109-28c9c769fc4a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:11.681940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c57752-7eb3-4d64-a109-28c9c769fc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:11.704233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-84c57752-7eb3-4d64-a109-28c9c769fc4a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:11.705457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-84c57752-7eb3-4d64-a109-28c9c769fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:11.784335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 573/1148] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:11.807229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355ac89-9d48-4d9a-9748-99c96df5b1a0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:11.908167 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:11.928943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355ac89-9d48-4d9a-9748-99c96df5b1a0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:11.965781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162c51-63da-4603-81b7-3297a1c19ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:11.966577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162c51-63da-4603-81b7-3297a1c19ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:11.966856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162c51-63da-4603-81b7-3297a1c19ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:11.967131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77162c51-63da-4603-81b7-3297a1c19ae6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:11.977779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77162c51-63da-4603-81b7-3297a1c19ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-77162c51-63da-4603-81b7-3297a1c19ae6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:11:11.985279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:11:11.986342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:11:11.987827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 576/1149] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:11:11.995083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c57752-7eb3-4d64-a109-28c9c769fc4a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:12.028753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:12.099418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c57752-7eb3-4d64-a109-28c9c769fc4a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:12.135051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:12.135709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:12.135991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:12.136265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:12.137016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:12.137353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:12.138720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:12.146980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 574/1150] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:12.152976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162c51-63da-4603-81b7-3297a1c19ae6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:12.264037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77162c51-63da-4603-81b7-3297a1c19ae6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:12.299129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75463c4b-c8f9-4867-a9c5-5a09c882c0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:12.300086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463c4b-c8f9-4867-a9c5-5a09c882c0ad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:12.300341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463c4b-c8f9-4867-a9c5-5a09c882c0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:12.300617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75463c4b-c8f9-4867-a9c5-5a09c882c0ad None admin] PATH_INFO: `/v3/users/8777022683b5416491d4fbd5fe27d3e7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:12.301962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75463c4b-c8f9-4867-a9c5-5a09c882c0ad None admin] RBAC: Authorizing `identity:get_user(user_id=8777022683b5416491d4fbd5fe27d3e7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:12.303410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75463c4b-c8f9-4867-a9c5-5a09c882c0ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:12.305053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 577/1151] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:11:12 2022] GET /identity/v3/users/8777022683b5416491d4fbd5fe27d3e7 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:12.313885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:12.414853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:12.424521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a846186-e557-4202-9ad1-dc1b57a9da68 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:12.466227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0afb80-c322-4ce8-a8cb-44f1b1604662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:12.467083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0afb80-c322-4ce8-a8cb-44f1b1604662 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:12.467362 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0afb80-c322-4ce8-a8cb-44f1b1604662 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:12.467635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0afb80-c322-4ce8-a8cb-44f1b1604662 None admin] PATH_INFO: `/v3/projects/ed7610999761466eb9edccfe7cc0577a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:12.468852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0afb80-c322-4ce8-a8cb-44f1b1604662 None admin] RBAC: Authorizing `identity:get_project(project_id=ed7610999761466eb9edccfe7cc0577a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:12.470309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0afb80-c322-4ce8-a8cb-44f1b1604662 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:12.472050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 575/1152] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:11:12 2022] GET /identity/v3/projects/ed7610999761466eb9edccfe7cc0577a => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:11:12.478383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75463c4b-c8f9-4867-a9c5-5a09c882c0ad None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:12.501052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:12.587064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75463c4b-c8f9-4867-a9c5-5a09c882c0ad None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:12.625605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ba7bcc-a319-443c-9557-8efc5fae6cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:12.626499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba7bcc-a319-443c-9557-8efc5fae6cf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:12.626779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba7bcc-a319-443c-9557-8efc5fae6cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:12.627053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ba7bcc-a319-443c-9557-8efc5fae6cf8 None admin] PATH_INFO: `/v3/projects/ed7610999761466eb9edccfe7cc0577a/users/8777022683b5416491d4fbd5fe27d3e7/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:12.629204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba7bcc-a319-443c-9557-8efc5fae6cf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ed7610999761466eb9edccfe7cc0577a, user_id=8777022683b5416491d4fbd5fe27d3e7, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:12.630650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ba7bcc-a319-443c-9557-8efc5fae6cf8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:12.649619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 578/1153] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:11:12 2022] PUT /identity/v3/projects/ed7610999761466eb9edccfe7cc0577a/users/8777022683b5416491d4fbd5fe27d3e7/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:11:14.783960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b9cc6-fe05-4019-b5cf-17e35cb21181 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:14.784736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b9cc6-fe05-4019-b5cf-17e35cb21181 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:14.784736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b9cc6-fe05-4019-b5cf-17e35cb21181 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:14.785944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 576/1154] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:14.792695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a440c-d383-4be5-b448-2b456ff8baa1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:14.793423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a440c-d383-4be5-b448-2b456ff8baa1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:14.793599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a440c-d383-4be5-b448-2b456ff8baa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:14.814695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-181a440c-d383-4be5-b448-2b456ff8baa1 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:14.816073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-181a440c-d383-4be5-b448-2b456ff8baa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:14.884738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 579/1155] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:14.911996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b9cc6-fe05-4019-b5cf-17e35cb21181 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:15.036259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b9cc6-fe05-4019-b5cf-17e35cb21181 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:15.074894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb5b83-3997-472e-9648-828c8237faf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:15.075740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5b83-3997-472e-9648-828c8237faf1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:15.076019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5b83-3997-472e-9648-828c8237faf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:15.076368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5b83-3997-472e-9648-828c8237faf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:15.088892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb5b83-3997-472e-9648-828c8237faf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-ecbb5b83-3997-472e-9648-828c8237faf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:11:15.097857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:11:15.098913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:11:15.099800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 577/1156] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:11:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:11:15.107622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a440c-d383-4be5-b448-2b456ff8baa1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:15.116252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:15.215605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a440c-d383-4be5-b448-2b456ff8baa1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:15.254511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:15.255172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:15.255453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:15.255726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:15.256476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:15.256780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:15.258180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:15.266650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 580/1157] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:11:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:15.273969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb5b83-3997-472e-9648-828c8237faf1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:15.304724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:15.381694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb5b83-3997-472e-9648-828c8237faf1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:15.415252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c94923-07e6-4a57-bb12-3e56221c46f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:15.416112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c94923-07e6-4a57-bb12-3e56221c46f3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:15.416392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c94923-07e6-4a57-bb12-3e56221c46f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:15.416668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c94923-07e6-4a57-bb12-3e56221c46f3 None admin] PATH_INFO: `/v3/users/8777022683b5416491d4fbd5fe27d3e7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:15.417913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c94923-07e6-4a57-bb12-3e56221c46f3 None admin] RBAC: Authorizing `identity:get_user(user_id=8777022683b5416491d4fbd5fe27d3e7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:15.419348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c94923-07e6-4a57-bb12-3e56221c46f3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:15.420973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 578/1158] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:11:15 2022] GET /identity/v3/users/8777022683b5416491d4fbd5fe27d3e7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:15.429836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:15.499535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:15.533371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2688c06-3998-4779-8eb9-5ade2078cfb2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:15.568719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6338af0-576c-4db2-8702-2ddd560de95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:15.569329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338af0-576c-4db2-8702-2ddd560de95b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:15.569611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338af0-576c-4db2-8702-2ddd560de95b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:15.569885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6338af0-576c-4db2-8702-2ddd560de95b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:15.642390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6338af0-576c-4db2-8702-2ddd560de95b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:15.643880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6338af0-576c-4db2-8702-2ddd560de95b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:15.716285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:15.756732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 581/1159] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:11:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:11:15.763043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c94923-07e6-4a57-bb12-3e56221c46f3 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:15.856391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:15.867680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c94923-07e6-4a57-bb12-3e56221c46f3 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:15.903566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175e111-444c-4ac3-bd5a-164e452c0969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:15.904428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175e111-444c-4ac3-bd5a-164e452c0969 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:15.904710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175e111-444c-4ac3-bd5a-164e452c0969 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:15.904987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175e111-444c-4ac3-bd5a-164e452c0969 None admin] PATH_INFO: `/v3/projects/ed7610999761466eb9edccfe7cc0577a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:15.906248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175e111-444c-4ac3-bd5a-164e452c0969 None admin] RBAC: Authorizing `identity:get_project(project_id=ed7610999761466eb9edccfe7cc0577a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:15.907687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175e111-444c-4ac3-bd5a-164e452c0969 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:15.909606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 579/1160] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:11:15 2022] GET /identity/v3/projects/ed7610999761466eb9edccfe7cc0577a => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:11:15.916624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6338af0-576c-4db2-8702-2ddd560de95b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:16.031267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6338af0-576c-4db2-8702-2ddd560de95b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:16.066055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762f397-c150-4089-8c72-4413ad673bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:16.066708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f397-c150-4089-8c72-4413ad673bde None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:16.066991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f397-c150-4089-8c72-4413ad673bde None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:16.067269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762f397-c150-4089-8c72-4413ad673bde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:16.068097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f397-c150-4089-8c72-4413ad673bde None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=ed7610999761466eb9edccfe7cc0577a, user.id=8777022683b5416491d4fbd5fe27d3e7) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:16.068411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f397-c150-4089-8c72-4413ad673bde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:16.069898 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762f397-c150-4089-8c72-4413ad673bde None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:16.076209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 582/1161] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:11:15 2022] GET /identity/v3/role_assignments?user.id=8777022683b5416491d4fbd5fe27d3e7&scope.project.id=ed7610999761466eb9edccfe7cc0577a&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:18.167329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f336f-d881-4871-b01a-b30e833284f0 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:18.167909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f336f-d881-4871-b01a-b30e833284f0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:18.167909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f336f-d881-4871-b01a-b30e833284f0 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:18.170194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 580/1162] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:18.179397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aedee3-d724-4bdd-a74d-2633f615c756 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:18.179792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aedee3-d724-4bdd-a74d-2633f615c756 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:18.179999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aedee3-d724-4bdd-a74d-2633f615c756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:18.201420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-31aedee3-d724-4bdd-a74d-2633f615c756 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:18.202612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-31aedee3-d724-4bdd-a74d-2633f615c756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:18.234293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:18.284405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 583/1163] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:18.319270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f336f-d881-4871-b01a-b30e833284f0 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:18.443873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f336f-d881-4871-b01a-b30e833284f0 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:18.480825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa653317-fe06-4881-971f-5c3f66d683b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:18.481830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa653317-fe06-4881-971f-5c3f66d683b8 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:18.482181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa653317-fe06-4881-971f-5c3f66d683b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:18.482539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa653317-fe06-4881-971f-5c3f66d683b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:18.520207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:18.572319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa653317-fe06-4881-971f-5c3f66d683b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:18.573798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa653317-fe06-4881-971f-5c3f66d683b8 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:18.675141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 581/1164] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:11:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:11:18.681453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aedee3-d724-4bdd-a74d-2633f615c756 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:18.787104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aedee3-d724-4bdd-a74d-2633f615c756 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:18.824691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ea0a6-79e0-42c3-a1f3-05388cd39717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:18.825528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ea0a6-79e0-42c3-a1f3-05388cd39717 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:18.825810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ea0a6-79e0-42c3-a1f3-05388cd39717 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:18.826082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ea0a6-79e0-42c3-a1f3-05388cd39717 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:18.833252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ea0a6-79e0-42c3-a1f3-05388cd39717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:18.836069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ea0a6-79e0-42c3-a1f3-05388cd39717 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-926ea0a6-79e0-42c3-a1f3-05388cd39717 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:11:18.850213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:11:18.851392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:11:18.852608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 584/1165] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:11:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:11:18.857640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa653317-fe06-4881-971f-5c3f66d683b8 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:18.917120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:18.966388 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa653317-fe06-4881-971f-5c3f66d683b8 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:19.006136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aad663-82cc-4ce4-8f8b-5247314c0c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:19.006789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aad663-82cc-4ce4-8f8b-5247314c0c1b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:19.007076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aad663-82cc-4ce4-8f8b-5247314c0c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:19.007340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aad663-82cc-4ce4-8f8b-5247314c0c1b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:19.008061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aad663-82cc-4ce4-8f8b-5247314c0c1b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:19.008367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aad663-82cc-4ce4-8f8b-5247314c0c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:19.009768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aad663-82cc-4ce4-8f8b-5247314c0c1b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:19.016893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 582/1166] 10.222.0.66 () {56 vars in 1115 bytes} [Thu Jun 16 17:11:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:21.124481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaba67-0145-4872-88a2-97153649e3ab None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:21.125080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaba67-0145-4872-88a2-97153649e3ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:21.125080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaba67-0145-4872-88a2-97153649e3ab None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:21.126475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 585/1167] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:21.132079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3f1f-d6bb-4c23-998a-9d559f24db59 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:21.132587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3f1f-d6bb-4c23-998a-9d559f24db59 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:21.132687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8f3f1f-d6bb-4c23-998a-9d559f24db59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:21.163081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0d8f3f1f-d6bb-4c23-998a-9d559f24db59 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:21.164286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0d8f3f1f-d6bb-4c23-998a-9d559f24db59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:21.255587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 583/1168] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:21.278862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaba67-0145-4872-88a2-97153649e3ab None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:21.314479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:21.398158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaba67-0145-4872-88a2-97153649e3ab None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:21.435321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85df0dc3-5ab1-4f18-a9e9-acc4603e7793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:21.435980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df0dc3-5ab1-4f18-a9e9-acc4603e7793 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:21.436255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df0dc3-5ab1-4f18-a9e9-acc4603e7793 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:21.436531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85df0dc3-5ab1-4f18-a9e9-acc4603e7793 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:21.437302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df0dc3-5ab1-4f18-a9e9-acc4603e7793 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:21.438667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85df0dc3-5ab1-4f18-a9e9-acc4603e7793 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:21.465717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 586/1169] 10.222.0.66 () {60 vars in 1138 bytes} [Thu Jun 16 17:11:21 2022] POST /identity/v3/domains => generated 267 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:11:23.589025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2c9d5-4ed0-456e-866a-20d54748d519 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:23.589965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2c9d5-4ed0-456e-866a-20d54748d519 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:23.589965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e2c9d5-4ed0-456e-866a-20d54748d519 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:23.592266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 584/1170] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:23.598625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc63fb9-075b-4f69-9ac1-ebd343d8d1ac None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:23.599367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc63fb9-075b-4f69-9ac1-ebd343d8d1ac None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:23.599521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc63fb9-075b-4f69-9ac1-ebd343d8d1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:23.628411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1cc63fb9-075b-4f69-9ac1-ebd343d8d1ac None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:23.629668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1cc63fb9-075b-4f69-9ac1-ebd343d8d1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:23.702672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 587/1171] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:23.733998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2c9d5-4ed0-456e-866a-20d54748d519 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:23.791054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:23.849735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e2c9d5-4ed0-456e-866a-20d54748d519 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:23.896096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e4ef94-d870-4a53-b5c1-198b0569d4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:23.896954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4ef94-d870-4a53-b5c1-198b0569d4cf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:23.897243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4ef94-d870-4a53-b5c1-198b0569d4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:23.897544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4ef94-d870-4a53-b5c1-198b0569d4cf None admin] PATH_INFO: `/v3/domains/79dcfa05b1664d948ed0b30bda461627` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:23.908205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4ef94-d870-4a53-b5c1-198b0569d4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=79dcfa05b1664d948ed0b30bda461627)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:23.909736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e4ef94-d870-4a53-b5c1-198b0569d4cf None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:23.911365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 585/1172] 10.222.0.66 () {56 vars in 1183 bytes} [Thu Jun 16 17:11:23 2022] GET /identity/v3/domains/79dcfa05b1664d948ed0b30bda461627 => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:23.917403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc63fb9-075b-4f69-9ac1-ebd343d8d1ac None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:23.933584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:24.032599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc63fb9-075b-4f69-9ac1-ebd343d8d1ac None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:24.069474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0796c3db-1616-4188-aa11-f64dcd9d15be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:24.070060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c3db-1616-4188-aa11-f64dcd9d15be None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:24.070343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c3db-1616-4188-aa11-f64dcd9d15be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:24.070624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0796c3db-1616-4188-aa11-f64dcd9d15be None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:24.071336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c3db-1616-4188-aa11-f64dcd9d15be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:24.072782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0796c3db-1616-4188-aa11-f64dcd9d15be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:24.102237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 588/1173] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:11:23 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:11:26.261582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dff0c-904b-40b8-ac82-20254172295f None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:26.262166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dff0c-904b-40b8-ac82-20254172295f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:26.262166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dff0c-904b-40b8-ac82-20254172295f None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:26.263482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 586/1174] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:26.269155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2102-78f5-423e-b728-1330e72155ff None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:26.269635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2102-78f5-423e-b728-1330e72155ff None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:26.269728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2102-78f5-423e-b728-1330e72155ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:26.289628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-172c2102-78f5-423e-b728-1330e72155ff None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:26.290826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-172c2102-78f5-423e-b728-1330e72155ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:26.359624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:26.372232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 589/1175] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:26.399582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dff0c-904b-40b8-ac82-20254172295f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:26.513382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dff0c-904b-40b8-ac82-20254172295f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:26.526447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:26.548479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9432065-56fe-4eda-838c-4972add75fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:26.549381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432065-56fe-4eda-838c-4972add75fde None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:26.549695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432065-56fe-4eda-838c-4972add75fde None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:26.550002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9432065-56fe-4eda-838c-4972add75fde None admin] PATH_INFO: `/v3/domains/79dcfa05b1664d948ed0b30bda461627` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:26.551230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9432065-56fe-4eda-838c-4972add75fde None admin] RBAC: Authorizing `identity:get_domain(domain_id=79dcfa05b1664d948ed0b30bda461627)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:26.552662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9432065-56fe-4eda-838c-4972add75fde None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:26.554315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 587/1176] 10.222.0.66 () {56 vars in 1183 bytes} [Thu Jun 16 17:11:26 2022] GET /identity/v3/domains/79dcfa05b1664d948ed0b30bda461627 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:26.560391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c2102-78f5-423e-b728-1330e72155ff None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:26.662797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c2102-78f5-423e-b728-1330e72155ff None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:26.698366 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33534973-49e2-4f75-8428-67ce384a6be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:26.699028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33534973-49e2-4f75-8428-67ce384a6be2 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:26.699309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33534973-49e2-4f75-8428-67ce384a6be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:26.699585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33534973-49e2-4f75-8428-67ce384a6be2 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:26.700307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33534973-49e2-4f75-8428-67ce384a6be2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:26.701776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33534973-49e2-4f75-8428-67ce384a6be2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:26.724467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 590/1177] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:11:26 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:11:28.845458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ba8c0-aba6-4348-a63a-8b1eee9c04df None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:28.845999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ba8c0-aba6-4348-a63a-8b1eee9c04df None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:28.845999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ba8c0-aba6-4348-a63a-8b1eee9c04df None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:28.847352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 588/1178] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:28.853701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f62e6-b5c1-431b-ac3a-e1946d59e4bc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:28.854409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f62e6-b5c1-431b-ac3a-e1946d59e4bc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:28.854562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f62e6-b5c1-431b-ac3a-e1946d59e4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:28.882156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d07f62e6-b5c1-431b-ac3a-e1946d59e4bc None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:28.883368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d07f62e6-b5c1-431b-ac3a-e1946d59e4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:28.936606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:28.960839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 591/1179] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:28.984111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ba8c0-aba6-4348-a63a-8b1eee9c04df None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:29.093979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ba8c0-aba6-4348-a63a-8b1eee9c04df None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:29.129902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175b5a0-e6fe-4230-a9ee-850e250cc4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:29.130608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175b5a0-e6fe-4230-a9ee-850e250cc4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:29.130895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175b5a0-e6fe-4230-a9ee-850e250cc4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:29.131167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175b5a0-e6fe-4230-a9ee-850e250cc4f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:29.141946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175b5a0-e6fe-4230-a9ee-850e250cc4f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7175b5a0-e6fe-4230-a9ee-850e250cc4f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:11:29.149539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:11:29.150649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:11:29.151724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 589/1180] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:11:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:11:29.160087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f62e6-b5c1-431b-ac3a-e1946d59e4bc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:29.264881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f62e6-b5c1-431b-ac3a-e1946d59e4bc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:29.306717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:29.307383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:29.307663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:29.307941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:29.308698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:29.309004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:29.310428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:29.321135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 592/1181] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:11:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:29.329693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175b5a0-e6fe-4230-a9ee-850e250cc4f5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:29.374867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:29.436569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175b5a0-e6fe-4230-a9ee-850e250cc4f5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:29.471766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a495b9b-c7a2-40c2-8cdf-6b39be48c391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:29.472629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a495b9b-c7a2-40c2-8cdf-6b39be48c391 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:29.472902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a495b9b-c7a2-40c2-8cdf-6b39be48c391 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:29.473204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a495b9b-c7a2-40c2-8cdf-6b39be48c391 None admin] PATH_INFO: `/v3/users/d3854ae228574162ba947c4357a85586` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:29.487328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a495b9b-c7a2-40c2-8cdf-6b39be48c391 None admin] RBAC: Authorizing `identity:get_user(user_id=d3854ae228574162ba947c4357a85586)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:29.488777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a495b9b-c7a2-40c2-8cdf-6b39be48c391 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:29.490497 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 590/1182] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:11:29 2022] GET /identity/v3/users/d3854ae228574162ba947c4357a85586 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:29.496564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:29.540185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:29.609559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2de442-ca40-4b23-8bd1-61569fc1b441 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:29.647911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932e90-9272-4753-b5b3-f2a26036012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:29.648558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932e90-9272-4753-b5b3-f2a26036012c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:29.648841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932e90-9272-4753-b5b3-f2a26036012c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:29.649118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77932e90-9272-4753-b5b3-f2a26036012c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:29.719157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932e90-9272-4753-b5b3-f2a26036012c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:29.720541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77932e90-9272-4753-b5b3-f2a26036012c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:29.765457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:29.825324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 593/1183] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:11:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:11:29.832094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a495b9b-c7a2-40c2-8cdf-6b39be48c391 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:29.942667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:29.946695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a495b9b-c7a2-40c2-8cdf-6b39be48c391 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:29.982575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755c8cf-e98c-4014-9936-a0f9691ad6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:29.983394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755c8cf-e98c-4014-9936-a0f9691ad6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:29.983674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755c8cf-e98c-4014-9936-a0f9691ad6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:29.983945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755c8cf-e98c-4014-9936-a0f9691ad6a2 None admin] PATH_INFO: `/v3/projects/2db8ad4586544aaa86431504103acfc7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:29.985231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755c8cf-e98c-4014-9936-a0f9691ad6a2 None admin] RBAC: Authorizing `identity:get_project(project_id=2db8ad4586544aaa86431504103acfc7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:29.986691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755c8cf-e98c-4014-9936-a0f9691ad6a2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:29.988638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 591/1184] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:11:29 2022] GET /identity/v3/projects/2db8ad4586544aaa86431504103acfc7 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:29.994899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932e90-9272-4753-b5b3-f2a26036012c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:30.105333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77932e90-9272-4753-b5b3-f2a26036012c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:30.146438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c93eb-2131-4f9f-ae06-96183c159ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:30.147070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c93eb-2131-4f9f-ae06-96183c159ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:30.147377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c93eb-2131-4f9f-ae06-96183c159ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:30.147651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c93eb-2131-4f9f-ae06-96183c159ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:30.148487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c93eb-2131-4f9f-ae06-96183c159ec7 None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=2db8ad4586544aaa86431504103acfc7, user.id=d3854ae228574162ba947c4357a85586) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:30.148798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c93eb-2131-4f9f-ae06-96183c159ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:30.150295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c93eb-2131-4f9f-ae06-96183c159ec7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:30.156092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 594/1185] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:11:29 2022] GET /identity/v3/role_assignments?user.id=d3854ae228574162ba947c4357a85586&scope.project.id=2db8ad4586544aaa86431504103acfc7&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:32.218298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34efb5-6c49-450e-9d98-17d3760f8049 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:32.218890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34efb5-6c49-450e-9d98-17d3760f8049 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:32.218890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34efb5-6c49-450e-9d98-17d3760f8049 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:32.220464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 592/1186] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:32.228212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d65f-d58d-444a-aa33-f4004018c525 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:32.228828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d65f-d58d-444a-aa33-f4004018c525 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:32.229504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f9d65f-d58d-444a-aa33-f4004018c525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:32.244939 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:32.251517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-03f9d65f-d58d-444a-aa33-f4004018c525 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:32.252812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-03f9d65f-d58d-444a-aa33-f4004018c525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:32.335652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 595/1187] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:32.357841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34efb5-6c49-450e-9d98-17d3760f8049 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:32.467002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34efb5-6c49-450e-9d98-17d3760f8049 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:32.513163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9b69-870c-480a-a813-a4747d6d6461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:32.513991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9b69-870c-480a-a813-a4747d6d6461 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:32.514277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9b69-870c-480a-a813-a4747d6d6461 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:32.514553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fc9b69-870c-480a-a813-a4747d6d6461 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:32.525075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fc9b69-870c-480a-a813-a4747d6d6461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c6fc9b69-870c-480a-a813-a4747d6d6461 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:11:32.532607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:11:32.533749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:11:32.534940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 593/1188] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:11:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:11:32.543028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9d65f-d58d-444a-aa33-f4004018c525 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:32.649196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f9d65f-d58d-444a-aa33-f4004018c525 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:32.694486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:32.695149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:32.695428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:32.695699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:32.696446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:32.696746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:32.698145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:32.700536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:32.707557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 596/1189] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:11:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:32.716981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9b69-870c-480a-a813-a4747d6d6461 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:32.750083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:32.827364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fc9b69-870c-480a-a813-a4747d6d6461 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:32.862248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68fbae-9ad4-4870-90e5-a7f18845fcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:32.863113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68fbae-9ad4-4870-90e5-a7f18845fcfb None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:32.863391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68fbae-9ad4-4870-90e5-a7f18845fcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:32.863671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68fbae-9ad4-4870-90e5-a7f18845fcfb None admin] PATH_INFO: `/v3/users/d3854ae228574162ba947c4357a85586` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:32.864941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68fbae-9ad4-4870-90e5-a7f18845fcfb None admin] RBAC: Authorizing `identity:get_user(user_id=d3854ae228574162ba947c4357a85586)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:32.866395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68fbae-9ad4-4870-90e5-a7f18845fcfb None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:32.868046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 594/1190] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:11:32 2022] GET /identity/v3/users/d3854ae228574162ba947c4357a85586 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:32.877260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:32.994682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325cf30-dcdc-4494-aa81-e07a0d39caf6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:33.030267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb1789-feff-4d2a-b70c-dcffc4a25ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:33.031061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1789-feff-4d2a-b70c-dcffc4a25ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:33.031338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1789-feff-4d2a-b70c-dcffc4a25ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:33.031607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb1789-feff-4d2a-b70c-dcffc4a25ee5 None admin] PATH_INFO: `/v3/projects/2db8ad4586544aaa86431504103acfc7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:33.032908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb1789-feff-4d2a-b70c-dcffc4a25ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=2db8ad4586544aaa86431504103acfc7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:33.034489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb1789-feff-4d2a-b70c-dcffc4a25ee5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:33.036183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 597/1191] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:11:32 2022] GET /identity/v3/projects/2db8ad4586544aaa86431504103acfc7 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:33.045327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68fbae-9ad4-4870-90e5-a7f18845fcfb None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:33.148810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:33.158275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68fbae-9ad4-4870-90e5-a7f18845fcfb None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:33.193959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c42da-5a25-47d7-9315-bf948b59c035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:33.194848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c42da-5a25-47d7-9315-bf948b59c035 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:33.195123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c42da-5a25-47d7-9315-bf948b59c035 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:33.195400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c42da-5a25-47d7-9315-bf948b59c035 None admin] PATH_INFO: `/v3/projects/2db8ad4586544aaa86431504103acfc7/users/d3854ae228574162ba947c4357a85586/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:33.197388 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c42da-5a25-47d7-9315-bf948b59c035 None admin] RBAC: Authorizing `identity:create_grant(project_id=2db8ad4586544aaa86431504103acfc7, user_id=d3854ae228574162ba947c4357a85586, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:33.198824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c42da-5a25-47d7-9315-bf948b59c035 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:33.219418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 595/1192] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:11:33 2022] PUT /identity/v3/projects/2db8ad4586544aaa86431504103acfc7/users/d3854ae228574162ba947c4357a85586/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:11:35.282161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2121-6d59-4212-b5e8-b423d99ee690 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:35.282722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2121-6d59-4212-b5e8-b423d99ee690 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:35.282722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2121-6d59-4212-b5e8-b423d99ee690 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:35.284108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 598/1193] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:11:35.290411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f6c5-7ec4-4fc7-b022-43a2378caf42 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:35.291148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f6c5-7ec4-4fc7-b022-43a2378caf42 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:35.291302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c753f6c5-7ec4-4fc7-b022-43a2378caf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:35.313421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c753f6c5-7ec4-4fc7-b022-43a2378caf42 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:11:35.314600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c753f6c5-7ec4-4fc7-b022-43a2378caf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:11:35.386982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 596/1194] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:11:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:11:35.419054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df2121-6d59-4212-b5e8-b423d99ee690 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:35.494895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:35.532677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df2121-6d59-4212-b5e8-b423d99ee690 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:35.567198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c12df-7d5b-445d-aca7-32661e90c736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:35.567987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c12df-7d5b-445d-aca7-32661e90c736 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:35.568275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c12df-7d5b-445d-aca7-32661e90c736 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:35.568544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c12df-7d5b-445d-aca7-32661e90c736 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:35.579522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c12df-7d5b-445d-aca7-32661e90c736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e31c12df-7d5b-445d-aca7-32661e90c736 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:11:35.587422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:11:35.588460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:11:35.589508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 599/1195] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:11:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:11:35.597436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f6c5-7ec4-4fc7-b022-43a2378caf42 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:35.697073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:35.720022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c753f6c5-7ec4-4fc7-b022-43a2378caf42 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:35.756751 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:35.757443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:35.757729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:35.758111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:35.758871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:35.759181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:35.760555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:35.769386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 597/1196] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:11:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:35.777909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c12df-7d5b-445d-aca7-32661e90c736 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:35.894877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c12df-7d5b-445d-aca7-32661e90c736 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:35.921567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:35.936733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93619d58-fba4-447c-b477-672fd5d76f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:35.938122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619d58-fba4-447c-b477-672fd5d76f95 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:35.938472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619d58-fba4-447c-b477-672fd5d76f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:35.938818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93619d58-fba4-447c-b477-672fd5d76f95 None admin] PATH_INFO: `/v3/users/d3854ae228574162ba947c4357a85586` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:35.940613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93619d58-fba4-447c-b477-672fd5d76f95 None admin] RBAC: Authorizing `identity:get_user(user_id=d3854ae228574162ba947c4357a85586)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:35.942777 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93619d58-fba4-447c-b477-672fd5d76f95 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:35.945310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 600/1197] 10.222.0.66 () {56 vars in 1177 bytes} [Thu Jun 16 17:11:35 2022] GET /identity/v3/users/d3854ae228574162ba947c4357a85586 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:11:35.954371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:36.062610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf287f7-8d3f-4048-8c2b-5a1906d48e54 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:36.097461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519a4b-5ea1-470b-9265-207b99516da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:36.098097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519a4b-5ea1-470b-9265-207b99516da6 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:36.098379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519a4b-5ea1-470b-9265-207b99516da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:36.098658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519a4b-5ea1-470b-9265-207b99516da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:36.115586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:36.170410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2519a4b-5ea1-470b-9265-207b99516da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:36.171783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2519a4b-5ea1-470b-9265-207b99516da6 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:36.276800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 598/1198] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:11:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:11:36.285610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93619d58-fba4-447c-b477-672fd5d76f95 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:36.386292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93619d58-fba4-447c-b477-672fd5d76f95 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:36.420957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33d51a-8959-4023-b541-4e5135e7a3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:36.421770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33d51a-8959-4023-b541-4e5135e7a3ee None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:36.422046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33d51a-8959-4023-b541-4e5135e7a3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:36.422318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33d51a-8959-4023-b541-4e5135e7a3ee None admin] PATH_INFO: `/v3/projects/2db8ad4586544aaa86431504103acfc7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:36.423540 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33d51a-8959-4023-b541-4e5135e7a3ee None admin] RBAC: Authorizing `identity:get_project(project_id=2db8ad4586544aaa86431504103acfc7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:36.425044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33d51a-8959-4023-b541-4e5135e7a3ee None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:36.426760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 601/1199] 10.222.0.66 () {56 vars in 1186 bytes} [Thu Jun 16 17:11:36 2022] GET /identity/v3/projects/2db8ad4586544aaa86431504103acfc7 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:36.433714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519a4b-5ea1-470b-9265-207b99516da6 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:11:36.488876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:36.543682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519a4b-5ea1-470b-9265-207b99516da6 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:11:36.589622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca0942-8b2e-42bf-805e-05703ffca7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:11:36.590272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca0942-8b2e-42bf-805e-05703ffca7de None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:11:36.590551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca0942-8b2e-42bf-805e-05703ffca7de None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:11:36.590821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca0942-8b2e-42bf-805e-05703ffca7de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:11:36.591646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca0942-8b2e-42bf-805e-05703ffca7de None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=2db8ad4586544aaa86431504103acfc7, user.id=d3854ae228574162ba947c4357a85586) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:11:36.591953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca0942-8b2e-42bf-805e-05703ffca7de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:11:36.593405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca0942-8b2e-42bf-805e-05703ffca7de None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:11:36.601777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 599/1200] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:11:36 2022] GET /identity/v3/role_assignments?user.id=d3854ae228574162ba947c4357a85586&scope.project.id=2db8ad4586544aaa86431504103acfc7&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:11:46.415938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=69 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:11:46.597158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=45 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:32.861043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8f72f-bce7-4a91-8688-c3d8e7f8bbd8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:32.861637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8f72f-bce7-4a91-8688-c3d8e7f8bbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:32.861637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8f72f-bce7-4a91-8688-c3d8e7f8bbd8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:32.863132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 602/1201] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:12:32.873275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3752cd-a208-4922-9546-9bc5f3210e8d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:32.873275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3752cd-a208-4922-9546-9bc5f3210e8d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:32.873275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3752cd-a208-4922-9546-9bc5f3210e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:32.891359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8f3752cd-a208-4922-9546-9bc5f3210e8d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:12:32.892876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8f3752cd-a208-4922-9546-9bc5f3210e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:12:32.976365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 600/1202] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:12:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:12:33.002273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8f72f-bce7-4a91-8688-c3d8e7f8bbd8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:33.142977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8f72f-bce7-4a91-8688-c3d8e7f8bbd8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:33.182645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:33.183449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:33.183729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:33.184002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:33.197330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:33.199360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:33.227736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:33.229994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:33.231054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:33.231054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:33.231054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:33.231054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:33.231054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 603/1203] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:33.236127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3752cd-a208-4922-9546-9bc5f3210e8d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:33.374632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3752cd-a208-4922-9546-9bc5f3210e8d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:33.414157 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:33.420861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1101bc-8a9c-4dea-8f70-b57b434bc8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:33.421849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1101bc-8a9c-4dea-8f70-b57b434bc8bf None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:33.422195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1101bc-8a9c-4dea-8f70-b57b434bc8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:33.422798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1101bc-8a9c-4dea-8f70-b57b434bc8bf None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:33.424024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1101bc-8a9c-4dea-8f70-b57b434bc8bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:33.424468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1101bc-8a9c-4dea-8f70-b57b434bc8bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:33.427244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1101bc-8a9c-4dea-8f70-b57b434bc8bf None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:33.447782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 601/1204] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:33.447970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:33.541138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:33.610183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dfdc2-4867-4bfc-b613-3ea0f8bea9a9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:33.654677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fce54f-222e-4b6b-bce6-76eaf0421754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:33.655414 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fce54f-222e-4b6b-bce6-76eaf0421754 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:33.655701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fce54f-222e-4b6b-bce6-76eaf0421754 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:33.655974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fce54f-222e-4b6b-bce6-76eaf0421754 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:33.656703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fce54f-222e-4b6b-bce6-76eaf0421754 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:33.658496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fce54f-222e-4b6b-bce6-76eaf0421754 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:33.683595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 604/1205] 10.222.0.66 () {60 vars in 1132 bytes} [Thu Jun 16 17:12:33 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:12:36.083806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebebca-7a8c-4f00-ba4f-24162e296167 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:36.084734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebebca-7a8c-4f00-ba4f-24162e296167 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:36.084734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eebebca-7a8c-4f00-ba4f-24162e296167 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:36.086081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 602/1206] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:12:36.092858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625153f2-bb54-42b0-83f7-c132a44a097e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:36.093474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625153f2-bb54-42b0-83f7-c132a44a097e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:36.093588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625153f2-bb54-42b0-83f7-c132a44a097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:36.112590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-625153f2-bb54-42b0-83f7-c132a44a097e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:12:36.113962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-625153f2-bb54-42b0-83f7-c132a44a097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:12:36.193355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 605/1207] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:12:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:12:36.217339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebebca-7a8c-4f00-ba4f-24162e296167 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:36.333389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eebebca-7a8c-4f00-ba4f-24162e296167 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:36.373675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f94ca-556f-43ba-b3dd-305670637310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:36.374559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f94ca-556f-43ba-b3dd-305670637310 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:36.374890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f94ca-556f-43ba-b3dd-305670637310 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:36.375202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f94ca-556f-43ba-b3dd-305670637310 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:36.386691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f94ca-556f-43ba-b3dd-305670637310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-fa5f94ca-556f-43ba-b3dd-305670637310 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:12:36.396016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:36.397003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:36.398401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 603/1208] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:12:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:36.405757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625153f2-bb54-42b0-83f7-c132a44a097e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:36.450373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:36.532119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625153f2-bb54-42b0-83f7-c132a44a097e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:36.574542 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:36.575340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:36.575586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:36.575984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:36.576822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:36.577161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:36.579133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:36.588655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 606/1209] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:36.594505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f94ca-556f-43ba-b3dd-305670637310 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:36.653786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:36.708909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f94ca-556f-43ba-b3dd-305670637310 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:36.746278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:36.747359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:36.747687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:36.747949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:36.763755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:36.765530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:36.784350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:36.785544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:36.791823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:36.791823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:36.791823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:36.791823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:36.791823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 604/1210] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:36.792181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:36.908901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:36.914315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de2d9d-fc76-400a-8d11-17887fd3a590 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:36.956418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:36.957094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:36.957437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:36.957729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:36.958503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:36.958856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:36.960591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:36.977134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 607/1211] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:36.986303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:37.101590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1890f-b1f6-4437-803f-7357ccd289b5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:37.103743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:37.143730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:37.144709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:37.144991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:37.145775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:37.155985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:37.156320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:12:37.175287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:37.176744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:37.177778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 605/1212] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:37.183345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:37.317235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293d65f-6bd1-43c1-8c4c-400928473eb5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:37.364182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:37.364864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:37.365144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:37.365458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:37.366955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:37.366955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:37.368561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:37.396579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 608/1213] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Jun 16 17:12:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:37.397894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:37.528831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d52fde-5f7a-4d08-a3de-bd73151e95a4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:37.569202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c11ef-0b95-4392-9ded-4951c8d8fb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:37.570010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c11ef-0b95-4392-9ded-4951c8d8fb8f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:37.570288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c11ef-0b95-4392-9ded-4951c8d8fb8f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:37.570683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c11ef-0b95-4392-9ded-4951c8d8fb8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:37.605660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:37.654969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c11ef-0b95-4392-9ded-4951c8d8fb8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:37.656720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c11ef-0b95-4392-9ded-4951c8d8fb8f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:37.773309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 606/1214] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:12:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:12:37.777140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:37.797901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:37.884391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9228d0-9b6d-449b-9628-eeb11e14512e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:37.920170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:37.921013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:37.921313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:37.921584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:37.933363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:37.935046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:37.954665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:37.955855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:37.956954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:37.956954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:37.956954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:37.956954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:37.956954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 609/1215] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:37.962185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c11ef-0b95-4392-9ded-4951c8d8fb8f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:38.029507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:38.077196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c11ef-0b95-4392-9ded-4951c8d8fb8f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:38.118073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:38.119005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:38.119178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:38.119412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:38.120237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:38.120644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:38.122357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:38.141484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 607/1216] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:38.142786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:38.225121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:38.266783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c739604-d813-4b9b-a3a2-865ab05e2491 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:38.307198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:38.308054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:38.308338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:38.308613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:38.316392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:38.317989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:12:38.333747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:38.340438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:38.341723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 610/1217] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:38.342260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:38.455215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1b5c0-0a05-43bf-bcd8-89e9b53f8f2f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:38.494353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:38.509253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542fc1ce-b178-4e9d-9744-f32f9d822e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:38.510080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fc1ce-b178-4e9d-9744-f32f9d822e89 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:38.510432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fc1ce-b178-4e9d-9744-f32f9d822e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:38.510770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542fc1ce-b178-4e9d-9744-f32f9d822e89 None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:38.511614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fc1ce-b178-4e9d-9744-f32f9d822e89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:38.512093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fc1ce-b178-4e9d-9744-f32f9d822e89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:38.513633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542fc1ce-b178-4e9d-9744-f32f9d822e89 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:38.533224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 608/1218] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:38.539937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:38.654673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351ce4a-c3b8-407e-999f-74f8e21369fa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:38.686915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:38.694820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebe225-fdf6-4dbf-ad18-515535bb4c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:38.695774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebe225-fdf6-4dbf-ad18-515535bb4c62 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:38.696089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebe225-fdf6-4dbf-ad18-515535bb4c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:38.696449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebe225-fdf6-4dbf-ad18-515535bb4c62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:38.697450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebe225-fdf6-4dbf-ad18-515535bb4c62 None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=7a2b40b05a6846099bd4c66609365ad5) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:38.697834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebe225-fdf6-4dbf-ad18-515535bb4c62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:38.699516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebe225-fdf6-4dbf-ad18-515535bb4c62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:38.717221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 611/1219] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:12:38 2022] GET /identity/v3/role_assignments?user.id=7a2b40b05a6846099bd4c66609365ad5&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:40.974644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e6b73-eec3-4964-8607-f9ad38d36021 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:40.975207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e6b73-eec3-4964-8607-f9ad38d36021 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:40.975207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4e6b73-eec3-4964-8607-f9ad38d36021 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:40.976607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 609/1220] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:12:40.982048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a31f5-b930-460a-9985-4213fe7e04d6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:40.982501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a31f5-b930-460a-9985-4213fe7e04d6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:40.982600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a31f5-b930-460a-9985-4213fe7e04d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:41.001586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b10a31f5-b930-460a-9985-4213fe7e04d6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:12:41.002942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b10a31f5-b930-460a-9985-4213fe7e04d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:12:41.073681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 612/1221] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:12:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:12:41.098501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e6b73-eec3-4964-8607-f9ad38d36021 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:41.221507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4e6b73-eec3-4964-8607-f9ad38d36021 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:41.264278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0492-78e6-43ff-b18a-ac92d390ae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:41.265025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0492-78e6-43ff-b18a-ac92d390ae58 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:41.265408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0492-78e6-43ff-b18a-ac92d390ae58 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:41.265759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5d0492-78e6-43ff-b18a-ac92d390ae58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:41.278308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5d0492-78e6-43ff-b18a-ac92d390ae58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-1b5d0492-78e6-43ff-b18a-ac92d390ae58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:12:41.286798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:41.288015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:41.288983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 610/1222] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:12:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:41.294304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a31f5-b930-460a-9985-4213fe7e04d6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:41.400869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a31f5-b930-460a-9985-4213fe7e04d6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:41.428920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:41.442142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9625333-abec-40a0-8831-64901593691d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:41.442828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9625333-abec-40a0-8831-64901593691d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:41.443139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9625333-abec-40a0-8831-64901593691d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:41.443434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9625333-abec-40a0-8831-64901593691d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:41.444411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9625333-abec-40a0-8831-64901593691d None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:41.444726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9625333-abec-40a0-8831-64901593691d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:41.446368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9625333-abec-40a0-8831-64901593691d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:41.455584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 613/1223] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:12:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:41.461769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0492-78e6-43ff-b18a-ac92d390ae58 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:41.510220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:41.575387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5d0492-78e6-43ff-b18a-ac92d390ae58 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:41.613427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:41.614251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:41.614551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:41.614809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:41.626387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:41.627974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:41.645117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:41.646301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:41.647470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:41.647470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:41.647470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:41.647470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:41.651078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 611/1224] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:41.652127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9625333-abec-40a0-8831-64901593691d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:41.764963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9625333-abec-40a0-8831-64901593691d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:41.802238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:41.802916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:41.803197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:41.803472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:41.804205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:41.804511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:41.805919 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:41.820426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 614/1225] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:41.827221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:41.938055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:41.947121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8c702-b659-45b7-9748-0d08ef3a40a7 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:41.985258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:41.986659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:41.986954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:41.987233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:41.995288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:41.996748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:12:42.011601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:42.012725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:42.013736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 612/1226] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:42.019903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:42.070708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:42.127893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4a4ce2-f1a6-4cb2-8143-43afc74a097d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:42.166668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:42.167464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:42.167746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:42.168017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:42.169115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:42.169457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:42.171020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:42.190224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 615/1227] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Jun 16 17:12:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:42.196556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:42.314047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f5aef-ad85-42d0-9c51-9787c1c47c80 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:42.353842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:42.354740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:42.355023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:42.355282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:42.368067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:42.369556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:42.388429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:42.389641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:42.390593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:42.390593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:42.390593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:42.390593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:42.390593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 613/1228] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:12:42.395512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:42.506551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f47b5e-c638-44c2-a973-cffde7ec68c5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:42.508845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:42.545770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:42.546445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:42.546728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:42.547004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:42.547829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:42.548138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:42.549538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:42.561474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 616/1229] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:42.568232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:42.577736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:42.679383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a40b2-6ed9-4596-8ca9-ccb7868c9df1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:42.715037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:42.715832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:42.716110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:42.716432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:42.723181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:42.724714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:12:42.739780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:42.740906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:42.742010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 614/1230] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 17:12:42.747065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:42.861146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f04b64-2a45-42b5-ab54-4b02cabfce6c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:42.895831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc80457-3760-4a1a-9c6e-cf339a7b5d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:42.896534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80457-3760-4a1a-9c6e-cf339a7b5d52 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:42.896803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80457-3760-4a1a-9c6e-cf339a7b5d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:42.897067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc80457-3760-4a1a-9c6e-cf339a7b5d52 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:42.897927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80457-3760-4a1a-9c6e-cf339a7b5d52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:42.898214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80457-3760-4a1a-9c6e-cf339a7b5d52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:42.899688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc80457-3760-4a1a-9c6e-cf339a7b5d52 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:42.913568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 617/1231] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:12:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:42.919771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:42.998045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:43.036819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c3f01-a830-4411-8925-51a993651e0f None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:43.075484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877c8cd-8f36-45eb-ae18-9852280f71c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:43.076381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877c8cd-8f36-45eb-ae18-9852280f71c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:43.076671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877c8cd-8f36-45eb-ae18-9852280f71c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:43.076944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877c8cd-8f36-45eb-ae18-9852280f71c9 None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/7a2b40b05a6846099bd4c66609365ad5/roles/3b84c0487e23407e8452c17b5886b640` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:43.092612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877c8cd-8f36-45eb-ae18-9852280f71c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=7a2b40b05a6846099bd4c66609365ad5, role_id=3b84c0487e23407e8452c17b5886b640)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:43.094122 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877c8cd-8f36-45eb-ae18-9852280f71c9 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:43.107312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 615/1232] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:12:42 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/7a2b40b05a6846099bd4c66609365ad5/roles/3b84c0487e23407e8452c17b5886b640 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:12:45.308021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d743a-72a3-4641-8bcc-9bd29b85bfed None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:45.308596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d743a-72a3-4641-8bcc-9bd29b85bfed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:45.308596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d743a-72a3-4641-8bcc-9bd29b85bfed None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:45.309968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 618/1233] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:12:45.315285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230eb30-4ed5-4bb8-ae1e-947c48708e7a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:45.315728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230eb30-4ed5-4bb8-ae1e-947c48708e7a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:45.315838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230eb30-4ed5-4bb8-ae1e-947c48708e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:45.333285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7230eb30-4ed5-4bb8-ae1e-947c48708e7a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:12:45.334889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7230eb30-4ed5-4bb8-ae1e-947c48708e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:12:45.412846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 616/1234] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:12:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:12:45.436202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d743a-72a3-4641-8bcc-9bd29b85bfed None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:45.467067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:45.554290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d743a-72a3-4641-8bcc-9bd29b85bfed None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:45.590371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc9780-f300-48e4-99b5-5d7591548ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:45.591104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc9780-f300-48e4-99b5-5d7591548ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:45.591390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc9780-f300-48e4-99b5-5d7591548ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:45.591664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc9780-f300-48e4-99b5-5d7591548ff0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:45.602692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc9780-f300-48e4-99b5-5d7591548ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b3fc9780-f300-48e4-99b5-5d7591548ff0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:12:45.611134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:45.612143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:45.613024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 619/1235] 10.222.0.66 () {56 vars in 1102 bytes} [Thu Jun 16 17:12:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:45.618640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230eb30-4ed5-4bb8-ae1e-947c48708e7a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:45.664775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:45.724921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230eb30-4ed5-4bb8-ae1e-947c48708e7a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:45.760396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:45.761071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:45.761373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:45.761653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:45.762405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:45.762709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:45.764063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:45.772229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 617/1236] 10.222.0.66 () {56 vars in 1103 bytes} [Thu Jun 16 17:12:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:45.778050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc9780-f300-48e4-99b5-5d7591548ff0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:45.873781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:45.893263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc9780-f300-48e4-99b5-5d7591548ff0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:45.927416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:45.928215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:45.928494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:45.928773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:45.940443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:45.941933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:45.960934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:45.962344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:45.963254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:45.963254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:45.963254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:45.963254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:45.963254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 620/1237] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:45.968486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:46.079686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10842d-abd1-4196-8efa-b5335530cff4 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:46.089121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:46.116657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:46.117460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:46.117815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:46.118128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:46.118971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:46.119335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:46.120803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:46.133592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 618/1238] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:12:46.139608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:46.248953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b0f3d-a80e-4c4a-b5bd-05c85c64ec84 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:46.295143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:46.296165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:46.296455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:46.296888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:46.300503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:46.306433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:46.308118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:12:46.322877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:46.323889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:46.324852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 621/1239] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:46.330510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:46.442339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36ba71-8c96-4981-9404-81706a95e40b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:46.479228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:46.479943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:46.480199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:46.480530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:46.481498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:46.481896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:46.483338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:46.505225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 619/1240] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Jun 16 17:12:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:12:46.511351 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:46.625040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c380757c-ad67-4dad-830c-4ec9454955de None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:46.672246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c1f1e-8e04-4c4b-b4bb-8b20666fe955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:46.673093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c1f1e-8e04-4c4b-b4bb-8b20666fe955 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:46.673481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c1f1e-8e04-4c4b-b4bb-8b20666fe955 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:46.673763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c1f1e-8e04-4c4b-b4bb-8b20666fe955 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:46.747052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:46.755465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c1f1e-8e04-4c4b-b4bb-8b20666fe955 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:46.757066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c1f1e-8e04-4c4b-b4bb-8b20666fe955 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:46.871541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 622/1241] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:12:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:12:46.878102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:46.915102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:47.002330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aff4ee-3209-4f79-aba5-9a33f1c98f52 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:47.039547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:47.040775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:47.041153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:47.041473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:47.052296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:47.053785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:47.073443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:47.074696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:47.075694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:47.075694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:47.075694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:47.075694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:47.075694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 620/1242] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:47.082883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c1f1e-8e04-4c4b-b4bb-8b20666fe955 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:47.188639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:47.197597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c1f1e-8e04-4c4b-b4bb-8b20666fe955 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:47.243938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:47.244577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:47.244904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:47.245246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:47.245961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:47.246258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:47.247707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:47.261801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 623/1243] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:47.267827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:47.349410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:47.374686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a32aa8d-15e9-4caf-92d1-11fdc3e607ba None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:47.417649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:47.418589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:47.418881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:47.419250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:47.426487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:47.427992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:12:47.443279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:47.444464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:47.445493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 621/1244] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:47.451164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:47.563237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01a1bd-da6b-404e-a065-a1934397d11f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:47.601551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8513e5-7b28-4d48-9849-77f1d24d22d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:47.602274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8513e5-7b28-4d48-9849-77f1d24d22d7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:47.602578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8513e5-7b28-4d48-9849-77f1d24d22d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:47.602905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8513e5-7b28-4d48-9849-77f1d24d22d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:47.603764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8513e5-7b28-4d48-9849-77f1d24d22d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:47.604074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8513e5-7b28-4d48-9849-77f1d24d22d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:47.605615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8513e5-7b28-4d48-9849-77f1d24d22d7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:47.619251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 624/1245] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:12:47.626690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:47.739387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b5571-6e8c-4f23-b35c-a54745d12ea5 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:47.783792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b763b-00e7-4a60-9a4f-03334b3d48d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:47.784583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b763b-00e7-4a60-9a4f-03334b3d48d2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:47.784891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b763b-00e7-4a60-9a4f-03334b3d48d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:47.785216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b763b-00e7-4a60-9a4f-03334b3d48d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:47.786106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b763b-00e7-4a60-9a4f-03334b3d48d2 None admin] RBAC: Adding query filter params (role.id=3b84c0487e23407e8452c17b5886b640, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=7a2b40b05a6846099bd4c66609365ad5) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:47.786394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b763b-00e7-4a60-9a4f-03334b3d48d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:47.788252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604b763b-00e7-4a60-9a4f-03334b3d48d2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:47.795653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 622/1246] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:12:47 2022] GET /identity/v3/role_assignments?user.id=7a2b40b05a6846099bd4c66609365ad5&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=3b84c0487e23407e8452c17b5886b640 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:49.967803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535e020-2e6f-43db-b64d-ab9590a2129e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:49.968404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535e020-2e6f-43db-b64d-ab9590a2129e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:49.968404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535e020-2e6f-43db-b64d-ab9590a2129e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:49.970025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 625/1247] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:12:49.975674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4919ec-50b4-4186-8478-62a35ef5b6a3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:49.976146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4919ec-50b4-4186-8478-62a35ef5b6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:49.976285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4919ec-50b4-4186-8478-62a35ef5b6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:49.981838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:49.995399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1a4919ec-50b4-4186-8478-62a35ef5b6a3 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:12:49.996672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1a4919ec-50b4-4186-8478-62a35ef5b6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:12:50.079814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 623/1248] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:12:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:12:50.113601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535e020-2e6f-43db-b64d-ab9590a2129e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:50.132294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:50.227899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535e020-2e6f-43db-b64d-ab9590a2129e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:50.272749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc7596-7b96-4e5c-8598-7d4ce30f0c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:50.273926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc7596-7b96-4e5c-8598-7d4ce30f0c6b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:50.274349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc7596-7b96-4e5c-8598-7d4ce30f0c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:50.274797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc7596-7b96-4e5c-8598-7d4ce30f0c6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:50.291041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc7596-7b96-4e5c-8598-7d4ce30f0c6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bfcc7596-7b96-4e5c-8598-7d4ce30f0c6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:12:50.302315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:50.303512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:50.304388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 626/1249] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:12:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:50.311278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4919ec-50b4-4186-8478-62a35ef5b6a3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:50.427153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4919ec-50b4-4186-8478-62a35ef5b6a3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:50.458226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:50.465305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:50.466011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:50.466339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:50.466743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:50.467760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:50.468198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:50.469829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:50.480058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 624/1250] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:12:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:50.487025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc7596-7b96-4e5c-8598-7d4ce30f0c6b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:50.572583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:50.599815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc7596-7b96-4e5c-8598-7d4ce30f0c6b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:50.638764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:50.639699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:50.640122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:50.640420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:50.652663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:50.654246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:50.675400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:50.676490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:50.677477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:50.677477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:50.677477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:50.677477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:50.678230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 627/1251] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:50.684337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:50.799491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb444e0-9f30-44d2-b41a-33c688b63b88 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:50.839332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:50.840022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:50.840301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:50.840571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:50.841333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:50.841700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:50.843179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:50.856500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 625/1252] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:50.862842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:50.993420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af66f0-bd9f-4e12-a6f9-d43b0f48014b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:51.023989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:51.034299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:51.035331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:51.035647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:51.035991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:51.044469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:51.046349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:12:51.065473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:51.066613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:51.067799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 628/1253] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:51.073132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:51.140212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:51.187580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86150a-c4f7-4fd7-b267-6288887f77e0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:51.222044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:51.222723 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:51.223006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:51.223301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:51.224233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:51.224603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:51.226121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:51.252556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 626/1254] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Jun 16 17:12:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:51.253891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:51.372108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6241f0d4-2ba6-4cff-9926-8e2e07c293fe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:51.410392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58d7b5-0886-4852-aec2-a226dbd32c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:51.411171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58d7b5-0886-4852-aec2-a226dbd32c72 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:51.411525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58d7b5-0886-4852-aec2-a226dbd32c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:51.411733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58d7b5-0886-4852-aec2-a226dbd32c72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:51.476321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:51.492326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58d7b5-0886-4852-aec2-a226dbd32c72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:51.494088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58d7b5-0886-4852-aec2-a226dbd32c72 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:51.606419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 629/1255] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:12:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:12:51.612692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:51.725285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6bfa6-af39-479f-9169-8158d1c19d67 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:51.742720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:51.769646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:51.770965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:51.771549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:51.771972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:51.784478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:51.785963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:51.805205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:51.806429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:51.807594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:51.807594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:51.807594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:51.807594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:51.807594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 627/1256] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:51.812724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58d7b5-0886-4852-aec2-a226dbd32c72 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:51.920594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:51.934937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58d7b5-0886-4852-aec2-a226dbd32c72 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:51.970725 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:51.971396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:51.971674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:51.971950 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:51.972696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:51.972976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:51.974432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:51.986181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 630/1257] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:51.991997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:52.115355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91d2e0f-f90d-4d16-8735-75f3f25d7e61 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:52.152704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:52.153971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:52.154305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:52.154588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:52.161804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:52.163232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:12:52.177335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:52.178490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:52.179616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 628/1258] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:52.185246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:52.321354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ecbd5-8e66-4ef1-a952-67f79e924e36 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:52.360474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122c3e7-a617-4241-ae6e-3dd7cf7be8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:52.361476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122c3e7-a617-4241-ae6e-3dd7cf7be8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:52.361832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122c3e7-a617-4241-ae6e-3dd7cf7be8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:52.362113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122c3e7-a617-4241-ae6e-3dd7cf7be8e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:52.362999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122c3e7-a617-4241-ae6e-3dd7cf7be8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:52.363329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122c3e7-a617-4241-ae6e-3dd7cf7be8e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:52.364946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122c3e7-a617-4241-ae6e-3dd7cf7be8e5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:52.378090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 631/1259] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:12:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:52.384329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:52.414405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:52.498842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68e1500-dc01-43e6-932d-af7ac9e5a24b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:52.537279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7c2d8-629d-4931-b184-5589a923ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:52.537962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7c2d8-629d-4931-b184-5589a923ac9d None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:52.538224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7c2d8-629d-4931-b184-5589a923ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:52.538590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7c2d8-629d-4931-b184-5589a923ac9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:52.539966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7c2d8-629d-4931-b184-5589a923ac9d None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=7a2b40b05a6846099bd4c66609365ad5) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:52.540626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7c2d8-629d-4931-b184-5589a923ac9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:52.543460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7c2d8-629d-4931-b184-5589a923ac9d None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:52.551003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 629/1260] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:12:52 2022] GET /identity/v3/role_assignments?user.id=7a2b40b05a6846099bd4c66609365ad5&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:54.669579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0d4a0-36cd-4645-a4e9-888796c9f926 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:54.670543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0d4a0-36cd-4645-a4e9-888796c9f926 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:54.670543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0d4a0-36cd-4645-a4e9-888796c9f926 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:54.672845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 632/1261] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:12:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:12:54.678569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83190-c34a-4c15-8856-cd4b2a7dba89 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:54.679033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83190-c34a-4c15-8856-cd4b2a7dba89 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:54.679124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83190-c34a-4c15-8856-cd4b2a7dba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:54.697642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8fa83190-c34a-4c15-8856-cd4b2a7dba89 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:12:54.698817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8fa83190-c34a-4c15-8856-cd4b2a7dba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:12:54.772322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 630/1262] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:12:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 17:12:54.795037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0d4a0-36cd-4645-a4e9-888796c9f926 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:54.808668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:54.931817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0d4a0-36cd-4645-a4e9-888796c9f926 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:54.968229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3baf83-4f6c-4170-a308-d45ae3cdc28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:54.969090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3baf83-4f6c-4170-a308-d45ae3cdc28c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:54.969467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3baf83-4f6c-4170-a308-d45ae3cdc28c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:54.969757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3baf83-4f6c-4170-a308-d45ae3cdc28c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:54.981033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3baf83-4f6c-4170-a308-d45ae3cdc28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fd3baf83-4f6c-4170-a308-d45ae3cdc28c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:12:54.988645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:54.989749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:54.990779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 633/1263] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:12:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:54.996206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa83190-c34a-4c15-8856-cd4b2a7dba89 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:55.015973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:55.102314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa83190-c34a-4c15-8856-cd4b2a7dba89 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:55.142150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:55.142814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:55.143090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:55.143368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:55.144120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:55.144430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:55.145843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:55.154744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 631/1264] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:12:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:55.160732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3baf83-4f6c-4170-a308-d45ae3cdc28c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:55.230996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:55.273384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3baf83-4f6c-4170-a308-d45ae3cdc28c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:55.318331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:55.319194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:55.319541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:55.319880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:55.333247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:55.335312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:55.352735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:55.353785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:55.355041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:55.355041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:55.355041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:55.355041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:55.355041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 634/1265] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:55.359860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:55.449811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:55.470607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95e3be-9559-471f-896b-a5bb5e8f0be1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:55.505437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:55.506103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:55.506387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:55.506657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:55.507384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:55.507695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:55.509052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:55.520864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 632/1266] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:55.527195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:55.637873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65634e72-f172-4d4e-bb41-84475347dc39 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:55.657567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:55.681546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:55.682825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:55.683286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:55.683743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:55.693322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:55.694761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:12:55.709375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:55.710443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:55.711437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 635/1267] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:55.716508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:55.822505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299139f-7361-43a3-91d0-68231f7e04cc None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:55.866335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:55.866996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:55.867284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:55.867550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:55.868389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:55.868697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:55.870158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:55.879762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:55.890127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 633/1268] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Jun 16 17:12:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:55.896147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:56.015717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea58e71b-842e-4936-b9a4-feb658d14a00 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:56.052174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:56.052960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:56.053247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:56.053554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:56.064629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:56.066110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:56.083796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:56.085120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:56.086196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:56.086196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:56.086196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:56.086196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:56.086196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 636/1269] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:56.093062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:56.198633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c38760-296f-4d2d-895f-c4836185f0aa None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:56.242344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:56.243039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:56.243332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:56.243640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:56.244434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:56.244821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:56.246442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:56.261862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 634/1270] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:12:56.270521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:56.314256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:56.389201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7e523-346d-4d77-bc1c-05dcbb096442 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:56.425302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:56.426087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:56.426360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:56.426629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:56.434056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:56.435568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:12:56.450325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:56.451330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:12:56.452304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 637/1271] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:56.457531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:56.523482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:56.570957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e2835-50e4-4e71-8f55-a21ea164c51b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:56.608138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c944d7-81cf-4564-9ace-e6d0f5d20a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:56.608780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c944d7-81cf-4564-9ace-e6d0f5d20a5f None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:56.609062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c944d7-81cf-4564-9ace-e6d0f5d20a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:56.609360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c944d7-81cf-4564-9ace-e6d0f5d20a5f None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:56.610186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c944d7-81cf-4564-9ace-e6d0f5d20a5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:56.610488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c944d7-81cf-4564-9ace-e6d0f5d20a5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:56.611904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c944d7-81cf-4564-9ace-e6d0f5d20a5f None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:56.623867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 635/1272] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:56.630407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:56.728937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:56.739488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d68879-e876-44d8-bbaa-95bae583aed9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:56.778289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499af98-6668-49f2-a840-14d9a9e44a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:56.779195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499af98-6668-49f2-a840-14d9a9e44a8c None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:56.779472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499af98-6668-49f2-a840-14d9a9e44a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:56.779747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499af98-6668-49f2-a840-14d9a9e44a8c None admin] PATH_INFO: `/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/7a2b40b05a6846099bd4c66609365ad5/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:56.781819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499af98-6668-49f2-a840-14d9a9e44a8c None admin] RBAC: Authorizing `identity:create_grant(project_id=8deb4e225ab24d98bacff11a9f183d8c, user_id=7a2b40b05a6846099bd4c66609365ad5, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:56.783250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499af98-6668-49f2-a840-14d9a9e44a8c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:56.799877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 638/1273] 10.222.0.66 () {58 vars in 1439 bytes} [Thu Jun 16 17:12:56 2022] PUT /identity/v3/projects/8deb4e225ab24d98bacff11a9f183d8c/users/7a2b40b05a6846099bd4c66609365ad5/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:12:58.870068 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044f1f-c8fa-4ef9-b234-bba886e00865 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:58.870617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044f1f-c8fa-4ef9-b234-bba886e00865 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:58.870617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04044f1f-c8fa-4ef9-b234-bba886e00865 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:58.871910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 636/1274] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:12:58.878432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc9bcb-953d-434b-b1ba-2a9faa026af2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:58.879132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc9bcb-953d-434b-b1ba-2a9faa026af2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:58.879318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc9bcb-953d-434b-b1ba-2a9faa026af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:58.899190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-90dc9bcb-953d-434b-b1ba-2a9faa026af2 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:12:58.900363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-90dc9bcb-953d-434b-b1ba-2a9faa026af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:12:58.970173 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 639/1275] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:12:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:12:58.993307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04044f1f-c8fa-4ef9-b234-bba886e00865 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:59.099590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04044f1f-c8fa-4ef9-b234-bba886e00865 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:59.116968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:59.135198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f1eb-c925-4016-be9c-a10727903c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:59.135919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f1eb-c925-4016-be9c-a10727903c62 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:59.136203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f1eb-c925-4016-be9c-a10727903c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:59.136480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f1eb-c925-4016-be9c-a10727903c62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:59.146618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20f1eb-c925-4016-be9c-a10727903c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-7d20f1eb-c925-4016-be9c-a10727903c62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 17:12:59.154192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:59.155286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:59.156319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 637/1276] 10.222.0.66 () {56 vars in 1096 bytes} [Thu Jun 16 17:12:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:59.162336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc9bcb-953d-434b-b1ba-2a9faa026af2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:59.261053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:59.283914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc9bcb-953d-434b-b1ba-2a9faa026af2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:59.327128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:59.328097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:59.328548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:59.328864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:59.329711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:59.330023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:59.331392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:59.340253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 640/1277] 10.222.0.66 () {56 vars in 1099 bytes} [Thu Jun 16 17:12:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:59.346271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f1eb-c925-4016-be9c-a10727903c62 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:59.451664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f1eb-c925-4016-be9c-a10727903c62 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:59.488991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:59.489846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:59.490126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:59.490395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:59.502202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:59.503643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:12:59.520446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:59.521484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:59.522647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:12:59.522647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:12:59.522647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:12:59.522647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:59.522647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 638/1278] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:59.529209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:59.644651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0f57d-0e71-4d25-944a-2b3ea42a31b5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:59.680267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:59.680921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:59.681231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:59.681498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:59.682219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:12:59.682523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:59.683888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:59.696600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 641/1279] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:12:59.705156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:59.740191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:59.814538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79653fdf-1cd9-43b6-9c1e-d8e0b84a895e None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:12:59.855301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:12:59.856454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:12:59.856825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:12:59.857246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] PATH_INFO: `/v3/users/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:12:59.866139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:12:59.867580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 17:12:59.881909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:12:59.883306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:12:59.884325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 639/1280] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:12:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:12:59.889038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:12:59.893367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:12:59.997860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c20f3-73b4-4bc5-b1a9-bbdc95eb74e4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:00.034125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:00.034796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:00.035073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:00.035348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:00.036182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:00.036489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:00.037957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:00.055203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 642/1281] 10.222.0.66 () {56 vars in 1143 bytes} [Thu Jun 16 17:12:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:13:00.061323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:00.160446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:00.170818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1c89e-083b-4cf3-9065-867c572c5fac None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:00.207028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561144fe-be9c-416f-93a7-db4d4656e8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:00.207662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561144fe-be9c-416f-93a7-db4d4656e8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:00.207934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561144fe-be9c-416f-93a7-db4d4656e8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:00.208206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561144fe-be9c-416f-93a7-db4d4656e8f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:00.287180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561144fe-be9c-416f-93a7-db4d4656e8f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:00.288558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561144fe-be9c-416f-93a7-db4d4656e8f2 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:00.390060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:00.402442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 640/1282] 10.222.0.66 () {58 vars in 1282 bytes} [Thu Jun 16 17:13:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 17:13:00.409003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:00.485506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:00.531183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f6860-9994-4801-a61a-6d070c393b92 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:00.569268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:00.570010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:00.570295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:00.570570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:00.581789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:00.583227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 17:13:00.601883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:13:00.602907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:13:00.604103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 17:13:00.604103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 17:13:00.604103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 17:13:00.604103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:13:00.604103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 643/1283] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:13:00.608977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561144fe-be9c-416f-93a7-db4d4656e8f2 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:00.715875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561144fe-be9c-416f-93a7-db4d4656e8f2 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:00.753881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:00.754542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:00.754821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:00.755090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:00.755814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:00.756121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:00.757546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:00.769326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 641/1284] 10.222.0.66 () {56 vars in 1109 bytes} [Thu Jun 16 17:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:13:00.775253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:00.884535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8036ef-45a6-4d19-b089-be23934b31de None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:00.900159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:00.925528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:00.926321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:00.926604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:00.926880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:00.933988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:00.935421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 17:13:00.949489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:13:00.950524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 16 17:13:00.951461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 644/1285] 10.222.0.66 () {56 vars in 1111 bytes} [Thu Jun 16 17:13:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:13:00.959357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:01.013512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:01.068025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927b27f5-ebb0-4b64-937e-10782d0be7f0 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:01.103652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43629d1f-8bea-49ab-814f-1a055d5e05ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:01.104301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629d1f-8bea-49ab-814f-1a055d5e05ab None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:01.104581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629d1f-8bea-49ab-814f-1a055d5e05ab None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:01.104859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43629d1f-8bea-49ab-814f-1a055d5e05ab None admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:01.105670 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43629d1f-8bea-49ab-814f-1a055d5e05ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:01.105978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43629d1f-8bea-49ab-814f-1a055d5e05ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:01.107384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43629d1f-8bea-49ab-814f-1a055d5e05ab None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:01.119832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 642/1286] 10.222.0.66 () {56 vars in 1148 bytes} [Thu Jun 16 17:13:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:13:01.127520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:01.246040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f048c8-b628-4118-97a6-2fa4ddf420f7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:01.281537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e564def-e307-4e07-b178-575547cef961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:01.282181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e564def-e307-4e07-b178-575547cef961 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:01.282458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e564def-e307-4e07-b178-575547cef961 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:01.282731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e564def-e307-4e07-b178-575547cef961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:01.283557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e564def-e307-4e07-b178-575547cef961 None admin] RBAC: Adding query filter params (role.id=b68ef721a7d6428e8b3fd70743791b8d, scope.project.id=8deb4e225ab24d98bacff11a9f183d8c, user.id=7a2b40b05a6846099bd4c66609365ad5) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:01.283865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e564def-e307-4e07-b178-575547cef961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:01.285312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e564def-e307-4e07-b178-575547cef961 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:01.291226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 645/1287] 10.222.0.66 () {56 vars in 1374 bytes} [Thu Jun 16 17:13:01 2022] GET /identity/v3/role_assignments?user.id=7a2b40b05a6846099bd4c66609365ad5&scope.project.id=8deb4e225ab24d98bacff11a9f183d8c&role.id=b68ef721a7d6428e8b3fd70743791b8d => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:13:03.394302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2fff2-6ce2-4e23-ae79-1066b39903df None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:03.394859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2fff2-6ce2-4e23-ae79-1066b39903df None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:03.394859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2fff2-6ce2-4e23-ae79-1066b39903df None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:03.396218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 643/1288] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:13:03.402565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ca332-e6a1-46d6-be13-5c13dd0f8110 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:03.403334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ca332-e6a1-46d6-be13-5c13dd0f8110 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:03.403481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ca332-e6a1-46d6-be13-5c13dd0f8110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:03.424178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c83ca332-e6a1-46d6-be13-5c13dd0f8110 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:13:03.425429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c83ca332-e6a1-46d6-be13-5c13dd0f8110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:13:03.458383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:03.504544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 646/1289] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:13:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:13:03.526489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2fff2-6ce2-4e23-ae79-1066b39903df None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:03.649892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2fff2-6ce2-4e23-ae79-1066b39903df None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:03.660427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:03.691534 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:03.692335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:03.692611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:03.692884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:03.693568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:03.694935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:13:03.706892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:13:03.708074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:03.709721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 644/1290] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:13:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:13:03.715226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ca332-e6a1-46d6-be13-5c13dd0f8110 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:03.835560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ca332-e6a1-46d6-be13-5c13dd0f8110 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:03.873162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7226b914-74d8-4c98-b752-2d09abd3dc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:03.873851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226b914-74d8-4c98-b752-2d09abd3dc96 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:03.874132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226b914-74d8-4c98-b752-2d09abd3dc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:03.874408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7226b914-74d8-4c98-b752-2d09abd3dc96 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:03.875167 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226b914-74d8-4c98-b752-2d09abd3dc96 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:03.875559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226b914-74d8-4c98-b752-2d09abd3dc96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:03.876989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7226b914-74d8-4c98-b752-2d09abd3dc96 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:03.881868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 647/1291] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Jun 16 17:13:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:13:03.890913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:03.999110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868d246-6ebc-43f5-99ab-a419eb32d64b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:04.049048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01f723-6c67-4e56-9e11-284123a10fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:04.050490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01f723-6c67-4e56-9e11-284123a10fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:04.050927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01f723-6c67-4e56-9e11-284123a10fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:04.051353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01f723-6c67-4e56-9e11-284123a10fc3 None admin] PATH_INFO: `/v3/services` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:04.052498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01f723-6c67-4e56-9e11-284123a10fc3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:04.052978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01f723-6c67-4e56-9e11-284123a10fc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:04.055178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db01f723-6c67-4e56-9e11-284123a10fc3 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:04.062950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 645/1292] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Jun 16 17:13:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:13:06.153860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f49e-1267-44bc-b4f5-4e82d2e4ed66 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:06.154706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f49e-1267-44bc-b4f5-4e82d2e4ed66 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:06.154706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8f49e-1267-44bc-b4f5-4e82d2e4ed66 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:06.157051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 648/1293] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:13:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:13:06.165532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d196a10-86d9-4144-b92e-83cf3a94758e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:06.166183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d196a10-86d9-4144-b92e-83cf3a94758e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:06.166351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d196a10-86d9-4144-b92e-83cf3a94758e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:06.192070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4d196a10-86d9-4144-b92e-83cf3a94758e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:13:06.193353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4d196a10-86d9-4144-b92e-83cf3a94758e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:13:06.224909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:06.266254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 646/1294] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:13:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:13:06.289276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8f49e-1267-44bc-b4f5-4e82d2e4ed66 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:06.323948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:06.405133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd8f49e-1267-44bc-b4f5-4e82d2e4ed66 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:06.447898 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76374432-a673-4ccd-be65-34cc108e5bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:06.448573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76374432-a673-4ccd-be65-34cc108e5bf2 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:06.448854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76374432-a673-4ccd-be65-34cc108e5bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:06.449129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76374432-a673-4ccd-be65-34cc108e5bf2 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:06.449899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76374432-a673-4ccd-be65-34cc108e5bf2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:06.451267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76374432-a673-4ccd-be65-34cc108e5bf2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:06.466755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 649/1295] 10.222.0.66 () {60 vars in 1141 bytes} [Thu Jun 16 17:13:06 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:13:08.532210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad13ee-f626-41b2-bfa7-1933c5e10828 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:08.532783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad13ee-f626-41b2-bfa7-1933c5e10828 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:08.532783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad13ee-f626-41b2-bfa7-1933c5e10828 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:08.535020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 647/1296] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:13:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:13:08.543689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c528c4-c586-4f50-a681-444ce57929c6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:08.544386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c528c4-c586-4f50-a681-444ce57929c6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:08.544657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c528c4-c586-4f50-a681-444ce57929c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:08.568695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-11c528c4-c586-4f50-a681-444ce57929c6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:13:08.569916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-11c528c4-c586-4f50-a681-444ce57929c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:13:08.648095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 650/1297] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:13:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:13:08.670872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad13ee-f626-41b2-bfa7-1933c5e10828 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:08.782452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad13ee-f626-41b2-bfa7-1933c5e10828 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:08.819804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:08.820594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:08.820874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:08.821160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:08.821846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:08.823208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:13:08.835825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:13:08.836954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:08.838170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 648/1298] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:13:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:13:08.844624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c528c4-c586-4f50-a681-444ce57929c6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:08.865848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:08.948865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c528c4-c586-4f50-a681-444ce57929c6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:08.988233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:08.988887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:08.989161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:08.989469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:08.990171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:08.990470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:08.991816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:08.997231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 651/1299] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Jun 16 17:13:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:13:09.004176 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:09.041552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:09.119269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979e624-9f12-4205-b035-52e2f442422b None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:09.154466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2992255-7b1b-4fe8-a4e3-72d7ec1390c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:09.155120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2992255-7b1b-4fe8-a4e3-72d7ec1390c7 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:09.155407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2992255-7b1b-4fe8-a4e3-72d7ec1390c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:09.155684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2992255-7b1b-4fe8-a4e3-72d7ec1390c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:09.156550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2992255-7b1b-4fe8-a4e3-72d7ec1390c7 None admin] RBAC: Adding query filter params (interface=public, service_id=413acdd0c2784c15a02c50bf154d93ac, region_id=RegionOne) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:09.156862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2992255-7b1b-4fe8-a4e3-72d7ec1390c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:09.158267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2992255-7b1b-4fe8-a4e3-72d7ec1390c7 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:09.163460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 649/1300] 10.222.0.66 () {56 vars in 1251 bytes} [Thu Jun 16 17:13:09 2022] GET /identity/v3/endpoints?service_id=413acdd0c2784c15a02c50bf154d93ac&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:13:09.170475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:09.269969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:09.293908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a698ce73-bdd3-4618-be03-c883fe54a960 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:09.330899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9122ff8-e8c3-4235-94aa-d61cc0df5192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:09.331568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9122ff8-e8c3-4235-94aa-d61cc0df5192 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:09.331843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9122ff8-e8c3-4235-94aa-d61cc0df5192 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:09.332116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9122ff8-e8c3-4235-94aa-d61cc0df5192 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:09.332788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9122ff8-e8c3-4235-94aa-d61cc0df5192 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:09.334503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9122ff8-e8c3-4235-94aa-d61cc0df5192 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:09.341481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 652/1301] 10.222.0.66 () {56 vars in 1087 bytes} [Thu Jun 16 17:13:09 2022] GET /identity/v3/services => generated 2191 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:13:11.444167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e2ea-8dd1-44e1-8d11-48f0ec24c8e5 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:11.444766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e2ea-8dd1-44e1-8d11-48f0ec24c8e5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:11.444766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e2ea-8dd1-44e1-8d11-48f0ec24c8e5 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:11.446994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 650/1302] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:13:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:13:11.455951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a5894-27ea-4806-a005-31c6f9a790dd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:11.456352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a5894-27ea-4806-a005-31c6f9a790dd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:11.456556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376a5894-27ea-4806-a005-31c6f9a790dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:11.479433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-376a5894-27ea-4806-a005-31c6f9a790dd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:13:11.480796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-376a5894-27ea-4806-a005-31c6f9a790dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:13:11.550682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 653/1303] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:13:11 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 17:13:11.578553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e2ea-8dd1-44e1-8d11-48f0ec24c8e5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:11.678914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:11.687116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e2ea-8dd1-44e1-8d11-48f0ec24c8e5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:11.728475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:11.729588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:11.729971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:11.730348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:11.731336 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:11.733479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 17:13:11.752408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 17:13:11.753607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application with Lock( Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self._enter() Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application return self.creator() Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application created_value = creator( Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: ERROR keystone.server.flask.application Jun 16 17:13:11.754931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 651/1304] 10.222.0.66 () {56 vars in 1117 bytes} [Thu Jun 16 17:13:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 17:13:11.759821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a5894-27ea-4806-a005-31c6f9a790dd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:11.838816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:11.865466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376a5894-27ea-4806-a005-31c6f9a790dd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:11.900796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c26462-29f5-419e-a1f9-930fe3cd7d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:11.901482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c26462-29f5-419e-a1f9-930fe3cd7d01 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:11.901737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c26462-29f5-419e-a1f9-930fe3cd7d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:11.902012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c26462-29f5-419e-a1f9-930fe3cd7d01 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:11.902718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c26462-29f5-419e-a1f9-930fe3cd7d01 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:13:11.903018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c26462-29f5-419e-a1f9-930fe3cd7d01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:11.904393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c26462-29f5-419e-a1f9-930fe3cd7d01 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:11.909503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 654/1305] 10.222.0.66 () {56 vars in 1116 bytes} [Thu Jun 16 17:13:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:13:11.918767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:13:12.025651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13ee0c2-5dd2-43bf-b9fc-4db9e4fa97a1 None admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:13:12.060502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0903ddd-22ce-4b31-8d1e-c09d394ad37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:13:12.061137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0903ddd-22ce-4b31-8d1e-c09d394ad37e None admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:13:12.061449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0903ddd-22ce-4b31-8d1e-c09d394ad37e None admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:13:12.061724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0903ddd-22ce-4b31-8d1e-c09d394ad37e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:13:12.062470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0903ddd-22ce-4b31-8d1e-c09d394ad37e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:13:12.063862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0903ddd-22ce-4b31-8d1e-c09d394ad37e None admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:13:12.088018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 652/1306] 10.222.0.66 () {60 vars in 1144 bytes} [Thu Jun 16 17:13:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:13:21.906824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=30 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:13:22.083438 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:15.966632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaee3a1-6e02-47fa-9b36-23ae5cd4fad5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:15.969331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaee3a1-6e02-47fa-9b36-23ae5cd4fad5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:15.970582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaee3a1-6e02-47fa-9b36-23ae5cd4fad5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:15.979837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 655/1307] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:15.982487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e854-0b32-46dd-9b9f-c440bbf573bf None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:15.983463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e854-0b32-46dd-9b9f-c440bbf573bf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:15.984038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e854-0b32-46dd-9b9f-c440bbf573bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:16.087437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-daa4e854-0b32-46dd-9b9f-c440bbf573bf None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:16.089054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-daa4e854-0b32-46dd-9b9f-c440bbf573bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:16.225925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 653/1308] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:16.274476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118e66-3310-4c45-b750-f89024fdb728 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:16.274767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118e66-3310-4c45-b750-f89024fdb728 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:16.275059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51118e66-3310-4c45-b750-f89024fdb728 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:16.276612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 656/1309] 10.222.0.66 () {54 vars in 964 bytes} [Thu Jun 16 17:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:16.284981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317768-c6d0-4a20-b9b6-8a4dcffbcee5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:16.285244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317768-c6d0-4a20-b9b6-8a4dcffbcee5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:16.285546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40317768-c6d0-4a20-b9b6-8a4dcffbcee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:16.375241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-40317768-c6d0-4a20-b9b6-8a4dcffbcee5 None None] MFA Rules not processed for user `72fb6996e1974060b162070f8da365aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:16.376364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-40317768-c6d0-4a20-b9b6-8a4dcffbcee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:16.398422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:16.459999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 654/1310] 10.222.0.66 () {58 vars in 1061 bytes} [Thu Jun 16 17:14:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:16.469259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118e66-3310-4c45-b750-f89024fdb728 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:16.471207 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-51118e66-3310-4c45-b750-f89024fdb728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:16.597817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51118e66-3310-4c45-b750-f89024fdb728 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:16.684912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2ad03-6a20-4100-a704-9a3d1ff16a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:16.686013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2ad03-6a20-4100-a704-9a3d1ff16a8d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:16.686447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2ad03-6a20-4100-a704-9a3d1ff16a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:16.686868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2ad03-6a20-4100-a704-9a3d1ff16a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:16.688696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8fa2ad03-6a20-4100-a704-9a3d1ff16a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:16.747241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:16.808365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2ad03-6a20-4100-a704-9a3d1ff16a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:16.810083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2ad03-6a20-4100-a704-9a3d1ff16a8d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:16.830995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 657/1311] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:19.307236 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58615539-d3d4-4705-88df-bc8ac9e86286 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:19.307835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58615539-d3d4-4705-88df-bc8ac9e86286 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:19.307835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58615539-d3d4-4705-88df-bc8ac9e86286 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:19.309212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 655/1312] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:19.314779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eaf09-b58c-4cdc-b287-1d645a138ab6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:19.315212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eaf09-b58c-4cdc-b287-1d645a138ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:19.315435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8eaf09-b58c-4cdc-b287-1d645a138ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:19.398807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7a8eaf09-b58c-4cdc-b287-1d645a138ab6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:19.400016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7a8eaf09-b58c-4cdc-b287-1d645a138ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:19.505128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 658/1313] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:19.558212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca20de-ea52-4922-92f3-5d9ae24044f7 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:19.558495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca20de-ea52-4922-92f3-5d9ae24044f7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:19.558847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca20de-ea52-4922-92f3-5d9ae24044f7 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:19.560287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 656/1314] 10.222.0.66 () {54 vars in 964 bytes} [Thu Jun 16 17:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:19.566748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27af7-342a-469f-a243-5a80800d5d43 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:19.567032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27af7-342a-469f-a243-5a80800d5d43 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:19.567310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d27af7-342a-469f-a243-5a80800d5d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:19.577955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:19.648245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-13d27af7-342a-469f-a243-5a80800d5d43 None None] MFA Rules not processed for user `72fb6996e1974060b162070f8da365aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:19.649583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-13d27af7-342a-469f-a243-5a80800d5d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:19.735784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 659/1315] 10.222.0.66 () {58 vars in 1061 bytes} [Thu Jun 16 17:14:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:19.745152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca20de-ea52-4922-92f3-5d9ae24044f7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:19.746734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c3ca20de-ea52-4922-92f3-5d9ae24044f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:19.856661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca20de-ea52-4922-92f3-5d9ae24044f7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:19.931183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:19.934146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297e70-921b-418e-afb2-a9da11c16bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:19.934828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297e70-921b-418e-afb2-a9da11c16bab service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:19.935127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297e70-921b-418e-afb2-a9da11c16bab service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:19.935447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297e70-921b-418e-afb2-a9da11c16bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:19.937963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f3297e70-921b-418e-afb2-a9da11c16bab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:20.047123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297e70-921b-418e-afb2-a9da11c16bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:20.048588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297e70-921b-418e-afb2-a9da11c16bab service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:20.069994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 657/1316] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:22.766647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ad5b-f752-4391-86b1-a83741236bc7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:22.767182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ad5b-f752-4391-86b1-a83741236bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:22.767182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ad5b-f752-4391-86b1-a83741236bc7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:22.768478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 660/1317] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:14:22.774625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441fad-1fbe-4513-9295-3e62f205238d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:22.775070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441fad-1fbe-4513-9295-3e62f205238d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:22.775187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68441fad-1fbe-4513-9295-3e62f205238d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:22.851430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-68441fad-1fbe-4513-9295-3e62f205238d None None] MFA Rules not processed for user `0a40ced9f6d3493d817d4b9d965ce69b`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:22.852643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-68441fad-1fbe-4513-9295-3e62f205238d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:22.915673 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:22.969416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 658/1318] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:22 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:23.013646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ad5b-f752-4391-86b1-a83741236bc7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:23.015758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f92ad5b-f752-4391-86b1-a83741236bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:23.156830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ad5b-f752-4391-86b1-a83741236bc7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:23.159734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:23.236454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed94443-7799-4b9a-99ba-cb84981772c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:23.237103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94443-7799-4b9a-99ba-cb84981772c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:23.237412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94443-7799-4b9a-99ba-cb84981772c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:23.237682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed94443-7799-4b9a-99ba-cb84981772c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:23.239333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ed94443-7799-4b9a-99ba-cb84981772c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:23.387086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed94443-7799-4b9a-99ba-cb84981772c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:23.388532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed94443-7799-4b9a-99ba-cb84981772c2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:23.414726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 661/1319] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:23 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:26.268655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91646ad6-a33a-43b0-a891-b27195d5f86e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:26.269320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91646ad6-a33a-43b0-a891-b27195d5f86e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:26.269320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91646ad6-a33a-43b0-a891-b27195d5f86e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:26.270621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 659/1320] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:26.276471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe58d15-4fd1-423a-87ae-fc999d7cb8c0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:26.276910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe58d15-4fd1-423a-87ae-fc999d7cb8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:26.277208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe58d15-4fd1-423a-87ae-fc999d7cb8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:26.327742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:26.356028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dfe58d15-4fd1-423a-87ae-fc999d7cb8c0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:26.357300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dfe58d15-4fd1-423a-87ae-fc999d7cb8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:26.478123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 662/1321] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:26.517809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91646ad6-a33a-43b0-a891-b27195d5f86e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:26.519353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-91646ad6-a33a-43b0-a891-b27195d5f86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:26.641470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91646ad6-a33a-43b0-a891-b27195d5f86e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:26.734007 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baeaa08-61b4-4ec4-919e-feba2b8741d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:26.734942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeaa08-61b4-4ec4-919e-feba2b8741d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:26.735317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeaa08-61b4-4ec4-919e-feba2b8741d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:26.735596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baeaa08-61b4-4ec4-919e-feba2b8741d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:26.738212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4baeaa08-61b4-4ec4-919e-feba2b8741d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:26.749943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:26.858942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baeaa08-61b4-4ec4-919e-feba2b8741d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:26.860350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baeaa08-61b4-4ec4-919e-feba2b8741d7 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:26.878363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 660/1322] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:29.745299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62cd91-3843-42ba-8fe5-9e07db7ad6c1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:29.745825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62cd91-3843-42ba-8fe5-9e07db7ad6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:29.745825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a62cd91-3843-42ba-8fe5-9e07db7ad6c1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:29.747176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 663/1323] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:29.752561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345cb18-d1fa-4fec-ae36-f85407af5413 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:29.753003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345cb18-d1fa-4fec-ae36-f85407af5413 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:29.753108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345cb18-d1fa-4fec-ae36-f85407af5413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:29.828767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5345cb18-d1fa-4fec-ae36-f85407af5413 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:29.830028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5345cb18-d1fa-4fec-ae36-f85407af5413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:29.896429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:29.941512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 661/1324] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:29.977105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62cd91-3843-42ba-8fe5-9e07db7ad6c1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:29.978742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4a62cd91-3843-42ba-8fe5-9e07db7ad6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:30.087929 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a62cd91-3843-42ba-8fe5-9e07db7ad6c1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:30.100154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:30.163892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99667da6-1e02-4628-9f48-a2f8fa9a2646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:30.164642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99667da6-1e02-4628-9f48-a2f8fa9a2646 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:30.164923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99667da6-1e02-4628-9f48-a2f8fa9a2646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:30.165206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99667da6-1e02-4628-9f48-a2f8fa9a2646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:30.166855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99667da6-1e02-4628-9f48-a2f8fa9a2646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:30.279379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99667da6-1e02-4628-9f48-a2f8fa9a2646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:30.280840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99667da6-1e02-4628-9f48-a2f8fa9a2646 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:30.298908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 664/1325] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:32.442616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f72f22-fc6e-4570-a9d5-1cd7f34f5fcd None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:32.443191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f72f22-fc6e-4570-a9d5-1cd7f34f5fcd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:32.443191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f72f22-fc6e-4570-a9d5-1cd7f34f5fcd None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:32.444540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 662/1326] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:32.450131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3406ca-3264-4490-a1a8-c880b98dbf22 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:32.450569 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3406ca-3264-4490-a1a8-c880b98dbf22 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:32.450744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3406ca-3264-4490-a1a8-c880b98dbf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:32.505253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:32.524058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ce3406ca-3264-4490-a1a8-c880b98dbf22 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:32.525314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ce3406ca-3264-4490-a1a8-c880b98dbf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:32.625159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 665/1327] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:32.660273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f72f22-fc6e-4570-a9d5-1cd7f34f5fcd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:32.661953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b4f72f22-fc6e-4570-a9d5-1cd7f34f5fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:32.773677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f72f22-fc6e-4570-a9d5-1cd7f34f5fcd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:32.848260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f964b9-f1bc-4c5a-b108-84c65d84e56d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:32.848908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f964b9-f1bc-4c5a-b108-84c65d84e56d service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:32.849204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f964b9-f1bc-4c5a-b108-84c65d84e56d service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:32.849488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f964b9-f1bc-4c5a-b108-84c65d84e56d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:32.851105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-99f964b9-f1bc-4c5a-b108-84c65d84e56d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:32.867976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:32.974165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f964b9-f1bc-4c5a-b108-84c65d84e56d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:32.975608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f964b9-f1bc-4c5a-b108-84c65d84e56d service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:32.994332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 663/1328] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:35.162791 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d7ec2-392e-4b94-a8a2-bc0e4b65813f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:35.163394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d7ec2-392e-4b94-a8a2-bc0e4b65813f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:35.163394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d7ec2-392e-4b94-a8a2-bc0e4b65813f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:35.164804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 666/1329] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:35.170489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb141d66-4ed2-4356-b839-9dd003eeffa5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:35.170969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb141d66-4ed2-4356-b839-9dd003eeffa5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:35.171064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb141d66-4ed2-4356-b839-9dd003eeffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:35.237071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-fb141d66-4ed2-4356-b839-9dd003eeffa5 None None] MFA Rules not processed for user `0a40ced9f6d3493d817d4b9d965ce69b`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:35.238326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fb141d66-4ed2-4356-b839-9dd003eeffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:35.306319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:35.338870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 664/1330] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:35.375906 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d7ec2-392e-4b94-a8a2-bc0e4b65813f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:35.377488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-552d7ec2-392e-4b94-a8a2-bc0e4b65813f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:35.496087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d7ec2-392e-4b94-a8a2-bc0e4b65813f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:35.518267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:35.578918 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f66d10f-e390-4c7d-897d-427e92086f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:35.579571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66d10f-e390-4c7d-897d-427e92086f84 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:35.579855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66d10f-e390-4c7d-897d-427e92086f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:35.580128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f66d10f-e390-4c7d-897d-427e92086f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:35.581802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f66d10f-e390-4c7d-897d-427e92086f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:35.697111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66d10f-e390-4c7d-897d-427e92086f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:35.698605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f66d10f-e390-4c7d-897d-427e92086f84 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:35.719520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 667/1331] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:35 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:38.102526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c17f-bab2-47a9-ac65-ab141598ea42 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:38.103069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c17f-bab2-47a9-ac65-ab141598ea42 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:38.103069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4759c17f-bab2-47a9-ac65-ab141598ea42 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:38.104635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 665/1332] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:38.109870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4b7f6-39df-4f1a-b44a-74e45dd50550 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:38.110297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4b7f6-39df-4f1a-b44a-74e45dd50550 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:38.110389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4b7f6-39df-4f1a-b44a-74e45dd50550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:38.173795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:38.177501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1fa4b7f6-39df-4f1a-b44a-74e45dd50550 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:38.179270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1fa4b7f6-39df-4f1a-b44a-74e45dd50550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:38.282807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 668/1333] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:38.318670 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759c17f-bab2-47a9-ac65-ab141598ea42 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:38.320957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4759c17f-bab2-47a9-ac65-ab141598ea42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:38.434240 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4759c17f-bab2-47a9-ac65-ab141598ea42 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:38.511141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89d640-0894-4a17-886a-95895700b391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:38.511790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d640-0894-4a17-886a-95895700b391 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:38.512077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d640-0894-4a17-886a-95895700b391 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:38.512337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d640-0894-4a17-886a-95895700b391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:38.514023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-db89d640-0894-4a17-886a-95895700b391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:38.540766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:38.624386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89d640-0894-4a17-886a-95895700b391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:38.625805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89d640-0894-4a17-886a-95895700b391 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:38.645245 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 666/1334] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:40.766947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b4378-8c0a-40f6-bcfd-97a8fa1f51fe None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:40.767779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b4378-8c0a-40f6-bcfd-97a8fa1f51fe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:40.767779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b4378-8c0a-40f6-bcfd-97a8fa1f51fe None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:40.769867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 669/1335] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:40.775235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f595b-4919-4f10-8ec2-f27cf746e165 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:40.775656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f595b-4919-4f10-8ec2-f27cf746e165 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:40.775765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998f595b-4919-4f10-8ec2-f27cf746e165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:40.847784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-998f595b-4919-4f10-8ec2-f27cf746e165 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:40.848968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-998f595b-4919-4f10-8ec2-f27cf746e165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:40.926269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:40.950645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 667/1336] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:40.993961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b4378-8c0a-40f6-bcfd-97a8fa1f51fe None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:40.996365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b66b4378-8c0a-40f6-bcfd-97a8fa1f51fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:41.107207 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b4378-8c0a-40f6-bcfd-97a8fa1f51fe None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:41.135706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:41.182688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96372594-3eaa-4867-9cf0-cdd5c6557c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:41.183340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96372594-3eaa-4867-9cf0-cdd5c6557c55 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:41.183626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96372594-3eaa-4867-9cf0-cdd5c6557c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:41.183901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96372594-3eaa-4867-9cf0-cdd5c6557c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:41.185633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-96372594-3eaa-4867-9cf0-cdd5c6557c55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:41.320246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96372594-3eaa-4867-9cf0-cdd5c6557c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:41.321726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96372594-3eaa-4867-9cf0-cdd5c6557c55 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:41.340523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 670/1337] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:43.770010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298de0-7560-4344-9d27-9e1a24a35eb3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:43.770542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298de0-7560-4344-9d27-9e1a24a35eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:43.770647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35298de0-7560-4344-9d27-9e1a24a35eb3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:43.772864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 668/1338] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:43.781857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd3fa1-057f-4fdb-a4bb-067efc23cccd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:43.782266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd3fa1-057f-4fdb-a4bb-067efc23cccd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:43.782455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bd3fa1-057f-4fdb-a4bb-067efc23cccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:43.861472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-07bd3fa1-057f-4fdb-a4bb-067efc23cccd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:43.862673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07bd3fa1-057f-4fdb-a4bb-067efc23cccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:43.871893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:43.971945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 671/1339] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:44.018915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35298de0-7560-4344-9d27-9e1a24a35eb3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:44.021305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-35298de0-7560-4344-9d27-9e1a24a35eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:44.146041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35298de0-7560-4344-9d27-9e1a24a35eb3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:44.222669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e9dfb-519c-4797-8868-544ac9162581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:44.223314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e9dfb-519c-4797-8868-544ac9162581 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:44.223588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e9dfb-519c-4797-8868-544ac9162581 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:44.223862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e9dfb-519c-4797-8868-544ac9162581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:44.225523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f18e9dfb-519c-4797-8868-544ac9162581 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:44.265116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:44.340456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e9dfb-519c-4797-8868-544ac9162581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:44.341868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e9dfb-519c-4797-8868-544ac9162581 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:44.361296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 669/1340] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:48.424894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eb825-30cf-4dbf-9201-acc841f0f1a9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:48.425469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eb825-30cf-4dbf-9201-acc841f0f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:48.425469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eb825-30cf-4dbf-9201-acc841f0f1a9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:48.426826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 672/1341] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:48.432598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cabeb6-abbf-45d1-a457-d5de902fbc3e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:48.433091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cabeb6-abbf-45d1-a457-d5de902fbc3e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:48.433454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cabeb6-abbf-45d1-a457-d5de902fbc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:48.509016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-19cabeb6-abbf-45d1-a457-d5de902fbc3e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:48.510254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-19cabeb6-abbf-45d1-a457-d5de902fbc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:48.608652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:48.622733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 670/1342] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:48.661423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eb825-30cf-4dbf-9201-acc841f0f1a9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:48.663826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-146eb825-30cf-4dbf-9201-acc841f0f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:48.762819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eb825-30cf-4dbf-9201-acc841f0f1a9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:48.767311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67aa12a-02a7-47c6-9003-05c0e9ee831f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:48.767961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa12a-02a7-47c6-9003-05c0e9ee831f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:48.768240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa12a-02a7-47c6-9003-05c0e9ee831f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:48.768511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67aa12a-02a7-47c6-9003-05c0e9ee831f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:48.778079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aa12a-02a7-47c6-9003-05c0e9ee831f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:48.779768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67aa12a-02a7-47c6-9003-05c0e9ee831f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:48.789665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 673/1343] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:51.613865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91effe-fe76-4f15-a29b-dcdca033b816 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:51.614429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91effe-fe76-4f15-a29b-dcdca033b816 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:51.614429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91effe-fe76-4f15-a29b-dcdca033b816 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:51.615762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 671/1344] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:51.621771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7965e-16dd-43b7-88f4-9a01a5f802a9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:51.622406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7965e-16dd-43b7-88f4-9a01a5f802a9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:51.622406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7965e-16dd-43b7-88f4-9a01a5f802a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:51.652984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-99f7965e-16dd-43b7-88f4-9a01a5f802a9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:51.654345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99f7965e-16dd-43b7-88f4-9a01a5f802a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:51.687616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:14:51.743982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 674/1345] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:51.781472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91effe-fe76-4f15-a29b-dcdca033b816 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:51.793490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91effe-fe76-4f15-a29b-dcdca033b816 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:51.797748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62186821-eb2c-4327-9e18-501b16a729e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:51.798422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62186821-eb2c-4327-9e18-501b16a729e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:51.798698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62186821-eb2c-4327-9e18-501b16a729e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:51.798954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62186821-eb2c-4327-9e18-501b16a729e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:51.808876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62186821-eb2c-4327-9e18-501b16a729e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:51.810402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62186821-eb2c-4327-9e18-501b16a729e9 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:51.819948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 672/1346] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:51 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:14:55.891854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9850-f8d2-4b91-9988-22878171982a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:55.892461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9850-f8d2-4b91-9988-22878171982a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:55.892461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae9850-f8d2-4b91-9988-22878171982a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:55.893820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 675/1347] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:14:55.899377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f722d0-5f91-471e-b748-51c478c13880 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:55.899896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f722d0-5f91-471e-b748-51c478c13880 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:55.900121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f722d0-5f91-471e-b748-51c478c13880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:55.919453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a7f722d0-5f91-471e-b748-51c478c13880 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:14:55.920603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a7f722d0-5f91-471e-b748-51c478c13880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:14:55.929396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 673/1348] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:14:55.967507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae9850-f8d2-4b91-9988-22878171982a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:14:55.979753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae9850-f8d2-4b91-9988-22878171982a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:14:55.983929 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0abbd9-e39b-4be5-8c0b-563182371587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:14:55.984575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0abbd9-e39b-4be5-8c0b-563182371587 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:14:55.984856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0abbd9-e39b-4be5-8c0b-563182371587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:14:55.985130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0abbd9-e39b-4be5-8c0b-563182371587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:14:55.994580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0abbd9-e39b-4be5-8c0b-563182371587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:14:55.996225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0abbd9-e39b-4be5-8c0b-563182371587 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:14:56.007095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 676/1349] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:14:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:01.077784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265735bb-9f8c-49bc-b9ab-df6db2b7897e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:01.078325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265735bb-9f8c-49bc-b9ab-df6db2b7897e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:01.078770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265735bb-9f8c-49bc-b9ab-df6db2b7897e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:01.080634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 674/1350] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:01.086771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1decd-007c-4108-aa53-cf96f231fe85 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:01.087610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1decd-007c-4108-aa53-cf96f231fe85 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:01.088091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1decd-007c-4108-aa53-cf96f231fe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:01.163004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cef1decd-007c-4108-aa53-cf96f231fe85 None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:01.164616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cef1decd-007c-4108-aa53-cf96f231fe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:01.249095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 677/1351] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:15:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:15:05.385624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd35fb-bca7-4e56-9065-5365dbb688e9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:05.385624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd35fb-bca7-4e56-9065-5365dbb688e9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:05.385624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd35fb-bca7-4e56-9065-5365dbb688e9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:05.386783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 675/1352] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:05.392130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e5c14-ff6f-464c-98b9-60ee71767c81 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:05.392600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e5c14-ff6f-464c-98b9-60ee71767c81 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:05.392701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e5c14-ff6f-464c-98b9-60ee71767c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:05.412091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9b2e5c14-ff6f-464c-98b9-60ee71767c81 None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:05.413422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9b2e5c14-ff6f-464c-98b9-60ee71767c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:05.423169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 678/1353] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:15:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:15:05.913655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=13 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:15:06.968356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f9073-3edb-4883-a67d-ee44e732cb94 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:06.968994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f9073-3edb-4883-a67d-ee44e732cb94 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:06.968994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f9073-3edb-4883-a67d-ee44e732cb94 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:06.970568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 676/1354] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:06.976000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070ad4-9630-4b07-aa3d-a70d91c20daf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:06.976543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070ad4-9630-4b07-aa3d-a70d91c20daf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:06.976640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52070ad4-9630-4b07-aa3d-a70d91c20daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:06.996364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-52070ad4-9630-4b07-aa3d-a70d91c20daf None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:06.997708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52070ad4-9630-4b07-aa3d-a70d91c20daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:07.007617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 679/1355] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:15:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:15:10.751076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd809f-f054-45d1-ae0c-8af3f23ef166 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:10.751630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd809f-f054-45d1-ae0c-8af3f23ef166 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:10.751630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd809f-f054-45d1-ae0c-8af3f23ef166 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:10.752871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 677/1356] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:10.758524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b110c4-ad5e-4863-ace0-e1fb44c800bf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:10.758954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b110c4-ad5e-4863-ace0-e1fb44c800bf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:10.759122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b110c4-ad5e-4863-ace0-e1fb44c800bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:10.778195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-37b110c4-ad5e-4863-ace0-e1fb44c800bf None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:10.779470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-37b110c4-ad5e-4863-ace0-e1fb44c800bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:10.787542 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 680/1357] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:10.844296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e5244-f867-41d3-b768-e6a74fad7194 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:10.845110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e5244-f867-41d3-b768-e6a74fad7194 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:10.845630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e5244-f867-41d3-b768-e6a74fad7194 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:10.847906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 678/1358] 10.222.0.66 () {54 vars in 961 bytes} [Thu Jun 16 17:15:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:10.858870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6121c-a0c4-4d01-bb25-84dd00e11e1f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:10.859459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6121c-a0c4-4d01-bb25-84dd00e11e1f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:10.859971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b6121c-a0c4-4d01-bb25-84dd00e11e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:10.909774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:15:10.917919 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a7b6121c-a0c4-4d01-bb25-84dd00e11e1f None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:10.919618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7b6121c-a0c4-4d01-bb25-84dd00e11e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:11.007440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 681/1359] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Jun 16 17:15:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:15:11.014959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e5244-f867-41d3-b768-e6a74fad7194 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:11.029851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e5244-f867-41d3-b768-e6a74fad7194 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:11.035374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8a00c-77b7-4acd-9727-e9aaed74ef5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:11.036121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8a00c-77b7-4acd-9727-e9aaed74ef5c service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:11.036503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8a00c-77b7-4acd-9727-e9aaed74ef5c service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:11.036877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8a00c-77b7-4acd-9727-e9aaed74ef5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:11.045733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8a00c-77b7-4acd-9727-e9aaed74ef5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:11.047466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8a00c-77b7-4acd-9727-e9aaed74ef5c service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:11.061666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 679/1360] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:13.384104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d7ff0-2c8c-48d4-be71-a981008670ae None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:13.385093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d7ff0-2c8c-48d4-be71-a981008670ae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:13.385433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d7ff0-2c8c-48d4-be71-a981008670ae None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:13.386937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 682/1361] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:13.393196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb32aa-3684-48f8-9852-cf848bdd7326 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:13.393781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb32aa-3684-48f8-9852-cf848bdd7326 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:13.393781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb32aa-3684-48f8-9852-cf848bdd7326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:13.414023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-eaeb32aa-3684-48f8-9852-cf848bdd7326 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:13.415038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-eaeb32aa-3684-48f8-9852-cf848bdd7326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:13.424055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 680/1362] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:13.470994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d7ff0-2c8c-48d4-be71-a981008670ae None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:13.495149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d7ff0-2c8c-48d4-be71-a981008670ae None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:13.501204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425da7b9-b737-4a52-ae46-a8197bc2c64d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:13.501984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da7b9-b737-4a52-ae46-a8197bc2c64d service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:13.502272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da7b9-b737-4a52-ae46-a8197bc2c64d service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:13.502551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425da7b9-b737-4a52-ae46-a8197bc2c64d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:13.513118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425da7b9-b737-4a52-ae46-a8197bc2c64d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:13.515142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425da7b9-b737-4a52-ae46-a8197bc2c64d service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:13.525756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 683/1363] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:16.058828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33598e2a-77f1-48bd-801d-c609795b87fe None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:16.059419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33598e2a-77f1-48bd-801d-c609795b87fe None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:16.059419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33598e2a-77f1-48bd-801d-c609795b87fe None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:16.061778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 681/1364] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:16.067566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a922fe-8a10-47ed-bb5e-a90096a42d86 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:16.068086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a922fe-8a10-47ed-bb5e-a90096a42d86 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:16.068086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a922fe-8a10-47ed-bb5e-a90096a42d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:16.089341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d3a922fe-8a10-47ed-bb5e-a90096a42d86 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:16.090719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d3a922fe-8a10-47ed-bb5e-a90096a42d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:16.100676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 684/1365] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:16.158209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1955-cfc4-4cd6-a69d-8acf4a0c128a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:16.158547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1955-cfc4-4cd6-a69d-8acf4a0c128a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:16.158741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c1955-cfc4-4cd6-a69d-8acf4a0c128a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:16.160365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 682/1366] 10.222.0.66 () {54 vars in 961 bytes} [Thu Jun 16 17:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:16.171749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7cf47-7599-46fe-8077-9d5e53cdfc6b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:16.172282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7cf47-7599-46fe-8077-9d5e53cdfc6b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:16.172711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7cf47-7599-46fe-8077-9d5e53cdfc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:16.193909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e6e7cf47-7599-46fe-8077-9d5e53cdfc6b None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:16.195153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e6e7cf47-7599-46fe-8077-9d5e53cdfc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:16.207043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 685/1367] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Jun 16 17:15:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:16.215133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c1955-cfc4-4cd6-a69d-8acf4a0c128a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:16.231038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c1955-cfc4-4cd6-a69d-8acf4a0c128a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:16.238647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b850ab-6264-4563-98bf-2ad3a7324c48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:16.239585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b850ab-6264-4563-98bf-2ad3a7324c48 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:16.239989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b850ab-6264-4563-98bf-2ad3a7324c48 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:16.240411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b850ab-6264-4563-98bf-2ad3a7324c48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:16.253373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b850ab-6264-4563-98bf-2ad3a7324c48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:16.256131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b850ab-6264-4563-98bf-2ad3a7324c48 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:16.268735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 683/1368] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:18.427102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921eaba-2b3f-45cc-8342-fc02b0dda62d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:18.427675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921eaba-2b3f-45cc-8342-fc02b0dda62d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:18.427675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921eaba-2b3f-45cc-8342-fc02b0dda62d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:18.429057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 686/1369] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:15:18.434594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0ad0-aa3b-41b5-a954-3b638cc2dd55 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:18.435076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0ad0-aa3b-41b5-a954-3b638cc2dd55 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:18.435156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f0ad0-aa3b-41b5-a954-3b638cc2dd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:18.454956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-640f0ad0-aa3b-41b5-a954-3b638cc2dd55 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:18.456493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-640f0ad0-aa3b-41b5-a954-3b638cc2dd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:18.465425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 684/1370] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:18.512776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921eaba-2b3f-45cc-8342-fc02b0dda62d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:18.528959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921eaba-2b3f-45cc-8342-fc02b0dda62d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:18.534845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f04b30-5009-4d26-ae3a-eb6de32c58c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:18.535521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f04b30-5009-4d26-ae3a-eb6de32c58c3 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:18.535805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f04b30-5009-4d26-ae3a-eb6de32c58c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:18.536071 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f04b30-5009-4d26-ae3a-eb6de32c58c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:18.546157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f04b30-5009-4d26-ae3a-eb6de32c58c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:18.547786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f04b30-5009-4d26-ae3a-eb6de32c58c3 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:18.558463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 687/1371] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:18.750185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f729-d4ea-4ff8-b151-0527a20c5664 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:18.750781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f729-d4ea-4ff8-b151-0527a20c5664 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:18.751022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee9f729-d4ea-4ff8-b151-0527a20c5664 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:18.752581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 685/1372] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:18.758718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc7761-06ae-4b90-8fed-39ede02e07d1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:18.759076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc7761-06ae-4b90-8fed-39ede02e07d1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:18.759355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc7761-06ae-4b90-8fed-39ede02e07d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:18.778694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e7dc7761-06ae-4b90-8fed-39ede02e07d1 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:18.780263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e7dc7761-06ae-4b90-8fed-39ede02e07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:18.793660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 688/1373] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:19.093324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cc423-eb92-4a3b-a659-8ff5db69fe6c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:19.093601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cc423-eb92-4a3b-a659-8ff5db69fe6c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:19.093874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cc423-eb92-4a3b-a659-8ff5db69fe6c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:19.095456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 686/1374] 10.222.0.66 () {54 vars in 963 bytes} [Thu Jun 16 17:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:19.100398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dafb0d-be5f-4da0-9aa8-f12d14126d48 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:19.100690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dafb0d-be5f-4da0-9aa8-f12d14126d48 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:19.100972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dafb0d-be5f-4da0-9aa8-f12d14126d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:19.152050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-33dafb0d-be5f-4da0-9aa8-f12d14126d48 None None] MFA Rules not processed for user `d6179dc2bd1d4ff2a04a8362d63e7b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:19.153992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-33dafb0d-be5f-4da0-9aa8-f12d14126d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:19.217688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 689/1375] 10.222.0.66 () {58 vars in 1060 bytes} [Thu Jun 16 17:15:19 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:19.225884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cc423-eb92-4a3b-a659-8ff5db69fe6c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:19.237665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280cc423-eb92-4a3b-a659-8ff5db69fe6c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:19.241576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f381f-858e-425e-b9a5-4449736acb3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:19.242468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f381f-858e-425e-b9a5-4449736acb3f service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:19.242945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f381f-858e-425e-b9a5-4449736acb3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:19.243411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f381f-858e-425e-b9a5-4449736acb3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:19.254281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f381f-858e-425e-b9a5-4449736acb3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:19.256170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f381f-858e-425e-b9a5-4449736acb3f service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:19.266992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 687/1376] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:20.747990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257971-735e-4e33-adb3-2ce8bdbf3567 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:20.748871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257971-735e-4e33-adb3-2ce8bdbf3567 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:20.749283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c257971-735e-4e33-adb3-2ce8bdbf3567 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:20.751557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 690/1377] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:20.758681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc25f8-b631-4290-8933-23a5ee4ec18c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:20.759174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc25f8-b631-4290-8933-23a5ee4ec18c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:20.759283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc25f8-b631-4290-8933-23a5ee4ec18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:20.779785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e4dc25f8-b631-4290-8933-23a5ee4ec18c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:20.781120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e4dc25f8-b631-4290-8933-23a5ee4ec18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:20.789937 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 688/1378] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:20.843557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c257971-735e-4e33-adb3-2ce8bdbf3567 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:20.864764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c257971-735e-4e33-adb3-2ce8bdbf3567 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:20.871622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2824370-c560-4379-86b7-9868a755104a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:20.872302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2824370-c560-4379-86b7-9868a755104a service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:20.872587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2824370-c560-4379-86b7-9868a755104a service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:20.872874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2824370-c560-4379-86b7-9868a755104a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:20.883205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2824370-c560-4379-86b7-9868a755104a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:20.884810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2824370-c560-4379-86b7-9868a755104a service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:20.889344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:15:20.897497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 691/1379] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:23.076922 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d515e-e6b4-4099-afcb-a6077323054b None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:23.077509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d515e-e6b4-4099-afcb-a6077323054b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:23.077745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91d515e-e6b4-4099-afcb-a6077323054b None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:23.079217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 689/1380] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:23.084815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610108-db0b-4c16-b350-312c243be730 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:23.085309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610108-db0b-4c16-b350-312c243be730 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:23.085409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d610108-db0b-4c16-b350-312c243be730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:23.105093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7d610108-db0b-4c16-b350-312c243be730 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:23.106467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7d610108-db0b-4c16-b350-312c243be730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:23.116175 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 692/1381] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:23.159749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d515e-e6b4-4099-afcb-a6077323054b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:23.174663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91d515e-e6b4-4099-afcb-a6077323054b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:23.180771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337448e3-fd31-4d16-a61a-332b691e8069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:23.181453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337448e3-fd31-4d16-a61a-332b691e8069 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:23.181738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337448e3-fd31-4d16-a61a-332b691e8069 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:23.182018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337448e3-fd31-4d16-a61a-332b691e8069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:23.191141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337448e3-fd31-4d16-a61a-332b691e8069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:23.192702 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337448e3-fd31-4d16-a61a-332b691e8069 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:23.203300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 690/1382] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:25.501043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84957b7-6c6a-4e8d-ad43-bd0d2bbafc67 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:25.501666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84957b7-6c6a-4e8d-ad43-bd0d2bbafc67 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:25.501666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84957b7-6c6a-4e8d-ad43-bd0d2bbafc67 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:25.502974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 693/1383] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:25.508773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf267a-5ec5-4098-8f12-a1b96767e224 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:25.509394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf267a-5ec5-4098-8f12-a1b96767e224 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:25.509633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf267a-5ec5-4098-8f12-a1b96767e224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:25.534469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f7bf267a-5ec5-4098-8f12-a1b96767e224 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:25.536336 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f7bf267a-5ec5-4098-8f12-a1b96767e224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:25.546404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 691/1384] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:25.593460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84957b7-6c6a-4e8d-ad43-bd0d2bbafc67 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:25.610046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84957b7-6c6a-4e8d-ad43-bd0d2bbafc67 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:25.615805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a440dfbf-ec03-4d80-bb9e-c4f3564319b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:25.616467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440dfbf-ec03-4d80-bb9e-c4f3564319b4 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:25.616750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440dfbf-ec03-4d80-bb9e-c4f3564319b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:25.617026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440dfbf-ec03-4d80-bb9e-c4f3564319b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:25.626773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440dfbf-ec03-4d80-bb9e-c4f3564319b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:25.629085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a440dfbf-ec03-4d80-bb9e-c4f3564319b4 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:25.643787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 694/1385] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:27.805678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903895a4-fcd8-48ee-b847-6b6093289c40 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:27.806330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903895a4-fcd8-48ee-b847-6b6093289c40 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:27.806330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903895a4-fcd8-48ee-b847-6b6093289c40 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:27.807594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 692/1386] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:27.813361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a74c-dab1-4e6c-bb41-d74e60ee7d27 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:27.813894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a74c-dab1-4e6c-bb41-d74e60ee7d27 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:27.814046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2a74c-dab1-4e6c-bb41-d74e60ee7d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:27.833719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8bd2a74c-dab1-4e6c-bb41-d74e60ee7d27 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:27.835320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8bd2a74c-dab1-4e6c-bb41-d74e60ee7d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:27.847200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 695/1387] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:27.892735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903895a4-fcd8-48ee-b847-6b6093289c40 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:27.909447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903895a4-fcd8-48ee-b847-6b6093289c40 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:27.915751 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a596ab2-d877-4d29-beab-435d5152b97e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:27.916430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596ab2-d877-4d29-beab-435d5152b97e service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:27.916724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596ab2-d877-4d29-beab-435d5152b97e service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:27.917006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596ab2-d877-4d29-beab-435d5152b97e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:27.926370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a596ab2-d877-4d29-beab-435d5152b97e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:27.928039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a596ab2-d877-4d29-beab-435d5152b97e service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:27.938834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 693/1388] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:30.113658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e84a9-553d-4471-af0f-e4630c3e8dd7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:30.114348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e84a9-553d-4471-af0f-e4630c3e8dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:30.114348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e84a9-553d-4471-af0f-e4630c3e8dd7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:30.115891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 696/1389] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:30.121522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf10ee5-b9a5-4ec3-b40e-e5e18a28be59 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:30.122020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf10ee5-b9a5-4ec3-b40e-e5e18a28be59 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:30.122020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf10ee5-b9a5-4ec3-b40e-e5e18a28be59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:30.140712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9cf10ee5-b9a5-4ec3-b40e-e5e18a28be59 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:30.142120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9cf10ee5-b9a5-4ec3-b40e-e5e18a28be59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:30.150332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 694/1390] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:30.196101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e84a9-553d-4471-af0f-e4630c3e8dd7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:30.212772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e84a9-553d-4471-af0f-e4630c3e8dd7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:30.220611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b78e5-ca15-4ce6-aab9-a3fb4d4a5151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:30.221503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b78e5-ca15-4ce6-aab9-a3fb4d4a5151 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:30.221850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b78e5-ca15-4ce6-aab9-a3fb4d4a5151 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:30.222238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b78e5-ca15-4ce6-aab9-a3fb4d4a5151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:30.233633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b78e5-ca15-4ce6-aab9-a3fb4d4a5151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:30.235722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b78e5-ca15-4ce6-aab9-a3fb4d4a5151 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:30.248600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 697/1391] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:32.439914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1ab84-2813-42cd-8317-e7912d09cf2a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:32.440517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1ab84-2813-42cd-8317-e7912d09cf2a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:32.440517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1ab84-2813-42cd-8317-e7912d09cf2a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:32.441913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 695/1392] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:15:32.447739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4a9b6-d416-4741-b510-892939c0686a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:32.448216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4a9b6-d416-4741-b510-892939c0686a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:32.448426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd4a9b6-d416-4741-b510-892939c0686a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:32.473838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-edd4a9b6-d416-4741-b510-892939c0686a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:32.476200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-edd4a9b6-d416-4741-b510-892939c0686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:32.487827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 698/1393] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:32.537694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1ab84-2813-42cd-8317-e7912d09cf2a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:32.553525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1ab84-2813-42cd-8317-e7912d09cf2a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:32.560713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6ba09-00de-442f-89f6-c50526bd42e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:32.561409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6ba09-00de-442f-89f6-c50526bd42e5 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:32.561695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6ba09-00de-442f-89f6-c50526bd42e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:32.561973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6ba09-00de-442f-89f6-c50526bd42e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:32.571574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b6ba09-00de-442f-89f6-c50526bd42e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:32.573227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b6ba09-00de-442f-89f6-c50526bd42e5 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:32.584592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 696/1394] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:34.727012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf017c0-4ee3-46f7-9bbb-6e8959c2fa1c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:34.727593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf017c0-4ee3-46f7-9bbb-6e8959c2fa1c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:34.727752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf017c0-4ee3-46f7-9bbb-6e8959c2fa1c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:34.729266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 699/1395] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:34.734610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5dc65-2e23-49ad-8100-213d3577a5e7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:34.735076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5dc65-2e23-49ad-8100-213d3577a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:34.735190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5dc65-2e23-49ad-8100-213d3577a5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:34.753304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-22c5dc65-2e23-49ad-8100-213d3577a5e7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:34.754427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-22c5dc65-2e23-49ad-8100-213d3577a5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:34.762943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 697/1396] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:34.806466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf017c0-4ee3-46f7-9bbb-6e8959c2fa1c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:34.822031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf017c0-4ee3-46f7-9bbb-6e8959c2fa1c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:34.827839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bcf0ee-b8e9-4d84-9c01-df4155739dd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:34.828519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcf0ee-b8e9-4d84-9c01-df4155739dd5 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:34.828802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcf0ee-b8e9-4d84-9c01-df4155739dd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:34.829077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bcf0ee-b8e9-4d84-9c01-df4155739dd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:34.838649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcf0ee-b8e9-4d84-9c01-df4155739dd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:34.840231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bcf0ee-b8e9-4d84-9c01-df4155739dd5 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:34.850428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 700/1397] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:34.877366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e6d7-3407-4661-9afa-40f2360293c3 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:34.878057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e6d7-3407-4661-9afa-40f2360293c3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:34.878722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7004e6d7-3407-4661-9afa-40f2360293c3 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:34.880551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 698/1398] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:34.889589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aa890-e240-400a-8eff-89b73b8ba94a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:34.889911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aa890-e240-400a-8eff-89b73b8ba94a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:34.890103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69aa890-e240-400a-8eff-89b73b8ba94a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:34.910279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a69aa890-e240-400a-8eff-89b73b8ba94a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:34.911497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a69aa890-e240-400a-8eff-89b73b8ba94a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:34.920878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 701/1399] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:34.980192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dc932-0517-4d89-b191-acd0586d4d5c None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:34.980429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dc932-0517-4d89-b191-acd0586d4d5c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:34.980731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dc932-0517-4d89-b191-acd0586d4d5c None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:34.982473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 699/1400] 10.222.0.66 () {54 vars in 962 bytes} [Thu Jun 16 17:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:34.988228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2be33-1adc-49d1-ba2a-213a5426f0de None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:34.988554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2be33-1adc-49d1-ba2a-213a5426f0de None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:34.988821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac2be33-1adc-49d1-ba2a-213a5426f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:35.044448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9ac2be33-1adc-49d1-ba2a-213a5426f0de None None] MFA Rules not processed for user `db842931b2394e8684012057347080af`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:35.045606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9ac2be33-1adc-49d1-ba2a-213a5426f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:35.111841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 702/1401] 10.222.0.66 () {58 vars in 1059 bytes} [Thu Jun 16 17:15:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:35.119393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dc932-0517-4d89-b191-acd0586d4d5c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:35.131372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dc932-0517-4d89-b191-acd0586d4d5c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:35.136016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94e3e4-f442-46c2-8d4b-0641a1d89193 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:35.136682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e3e4-f442-46c2-8d4b-0641a1d89193 service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:35.136969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e3e4-f442-46c2-8d4b-0641a1d89193 service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:35.137276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e3e4-f442-46c2-8d4b-0641a1d89193 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:35.146972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94e3e4-f442-46c2-8d4b-0641a1d89193 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:35.148428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94e3e4-f442-46c2-8d4b-0641a1d89193 service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:35.158471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 700/1402] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:35.877405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506f639-3702-4fbb-ba4d-22b3d9336f53 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:35.877911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506f639-3702-4fbb-ba4d-22b3d9336f53 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:35.878008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506f639-3702-4fbb-ba4d-22b3d9336f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:35.927330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e506f639-3702-4fbb-ba4d-22b3d9336f53 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:35.928534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e506f639-3702-4fbb-ba4d-22b3d9336f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:35.971439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:15:35.996275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 703/1403] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:15:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:36.036529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fd47f-5c34-4fe8-943a-7a4e3f9480a2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:36.037017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fd47f-5c34-4fe8-943a-7a4e3f9480a2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:36.037132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fd47f-5c34-4fe8-943a-7a4e3f9480a2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:36.038570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 701/1404] 10.222.0.66 () {54 vars in 942 bytes} [Thu Jun 16 17:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:36.045286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a4d7-d38b-44bc-a82d-049de1da86a7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:36.045579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a4d7-d38b-44bc-a82d-049de1da86a7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:36.045845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336a4d7-d38b-44bc-a82d-049de1da86a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:36.096598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7336a4d7-d38b-44bc-a82d-049de1da86a7 None None] MFA Rules not processed for user `0f15957f640a411eb4ac8861454f2b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:36.097813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7336a4d7-d38b-44bc-a82d-049de1da86a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:36.172558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 704/1405] 10.222.0.66 () {58 vars in 1039 bytes} [Thu Jun 16 17:15:36 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:15:36.183104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fd47f-5c34-4fe8-943a-7a4e3f9480a2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:36.190785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=61 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:15:36.197861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fd47f-5c34-4fe8-943a-7a4e3f9480a2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:36.203123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614962af-aa65-46d8-a90e-e8cf7b0fb1ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:36.203766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614962af-aa65-46d8-a90e-e8cf7b0fb1ea service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:36.204046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614962af-aa65-46d8-a90e-e8cf7b0fb1ea service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:36.204322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614962af-aa65-46d8-a90e-e8cf7b0fb1ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:36.212864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614962af-aa65-46d8-a90e-e8cf7b0fb1ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:36.214476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614962af-aa65-46d8-a90e-e8cf7b0fb1ea service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:36.222790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 702/1406] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:15:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:15:37.003879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ad516-d375-41ad-b784-48ff7083f2b3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:37.004417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ad516-d375-41ad-b784-48ff7083f2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:37.004417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ad516-d375-41ad-b784-48ff7083f2b3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:37.006205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 705/1407] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:37.011511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf2fc-2d1c-4663-b7d3-4c7778a85d66 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:37.011797 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf2fc-2d1c-4663-b7d3-4c7778a85d66 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:37.012070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcf2fc-2d1c-4663-b7d3-4c7778a85d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:37.035484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-34fcf2fc-2d1c-4663-b7d3-4c7778a85d66 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:37.036697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-34fcf2fc-2d1c-4663-b7d3-4c7778a85d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:37.054228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 703/1408] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:37.089691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ad516-d375-41ad-b784-48ff7083f2b3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:15:37.104836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ad516-d375-41ad-b784-48ff7083f2b3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:15:37.110461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33a7db-8bf1-4396-a7d3-83f4fb14eea3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:15:37.111224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33a7db-8bf1-4396-a7d3-83f4fb14eea3 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:37.111506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33a7db-8bf1-4396-a7d3-83f4fb14eea3 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:37.111781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33a7db-8bf1-4396-a7d3-83f4fb14eea3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:37.120675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33a7db-8bf1-4396-a7d3-83f4fb14eea3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:15:37.122431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33a7db-8bf1-4396-a7d3-83f4fb14eea3 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:15:37.134476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 706/1409] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:15:47.029615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:15:47.128390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=59 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:15:58.341828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ca685-faff-4d84-a439-e3be245b52f4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:58.342420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ca685-faff-4d84-a439-e3be245b52f4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:58.342420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ca685-faff-4d84-a439-e3be245b52f4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:58.343914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 704/1410] 10.222.0.66 () {54 vars in 918 bytes} [Thu Jun 16 17:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:58.349239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7accc-97d0-463c-a476-d8f935bd3ec6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:58.349651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7accc-97d0-463c-a476-d8f935bd3ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:58.349802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e7accc-97d0-463c-a476-d8f935bd3ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:58.369127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b7e7accc-97d0-463c-a476-d8f935bd3ec6 None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:58.370368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b7e7accc-97d0-463c-a476-d8f935bd3ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:58.380340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 707/1411] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Jun 16 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:15:58.394904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab96b6a-d452-4716-ac32-ff5f27b1ad1a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:58.395197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab96b6a-d452-4716-ac32-ff5f27b1ad1a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:58.395481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab96b6a-d452-4716-ac32-ff5f27b1ad1a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:58.396878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 705/1412] 10.222.0.66 () {54 vars in 918 bytes} [Thu Jun 16 17:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:15:58.401559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa46c0c-1f22-4e4c-ac91-3437320ccc47 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:15:58.401875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa46c0c-1f22-4e4c-ac91-3437320ccc47 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:15:58.402182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa46c0c-1f22-4e4c-ac91-3437320ccc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:15:58.422619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-efa46c0c-1f22-4e4c-ac91-3437320ccc47 None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:15:58.423838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-efa46c0c-1f22-4e4c-ac91-3437320ccc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:15:58.434153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 708/1413] 10.222.0.66 () {58 vars in 1015 bytes} [Thu Jun 16 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:00.999118 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062f7b-1143-4b3f-97ab-0a3f205fec38 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:00.999792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062f7b-1143-4b3f-97ab-0a3f205fec38 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:00.999792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95062f7b-1143-4b3f-97ab-0a3f205fec38 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:01.000922 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:16:01.007300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9ea6e-647b-469c-b693-87e0a2d85b61 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:01.007739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9ea6e-647b-469c-b693-87e0a2d85b61 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:01.007870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe9ea6e-647b-469c-b693-87e0a2d85b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:01.035519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6fe9ea6e-647b-469c-b693-87e0a2d85b61 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:01.036716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6fe9ea6e-647b-469c-b693-87e0a2d85b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:01.046215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:01.093550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95062f7b-1143-4b3f-97ab-0a3f205fec38 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:01.110134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95062f7b-1143-4b3f-97ab-0a3f205fec38 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:01.116466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39ef41-9f49-4e0e-bfcd-1456fe459b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:01.117110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ef41-9f49-4e0e-bfcd-1456fe459b64 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:01.117430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ef41-9f49-4e0e-bfcd-1456fe459b64 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:01.117703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39ef41-9f49-4e0e-bfcd-1456fe459b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:01.126267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39ef41-9f49-4e0e-bfcd-1456fe459b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:01.127859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39ef41-9f49-4e0e-bfcd-1456fe459b64 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:01.137135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 707/1416] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:02.100766 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4808a-7d23-4663-b5f4-7cda5bd6ae0f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:02.101404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4808a-7d23-4663-b5f4-7cda5bd6ae0f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:02.101404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4808a-7d23-4663-b5f4-7cda5bd6ae0f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:02.102697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:02.107772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e843a2-456d-4411-8f35-790df8dbd191 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:02.108203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e843a2-456d-4411-8f35-790df8dbd191 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:02.108575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e843a2-456d-4411-8f35-790df8dbd191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:02.136486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-86e843a2-456d-4411-8f35-790df8dbd191 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:02.138367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-86e843a2-456d-4411-8f35-790df8dbd191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:02.148159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:02.303528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4808a-7d23-4663-b5f4-7cda5bd6ae0f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:02.320678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4808a-7d23-4663-b5f4-7cda5bd6ae0f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:02.327447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27325e-9fe8-4694-bde5-b6394943f122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:02.328084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27325e-9fe8-4694-bde5-b6394943f122 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:02.328352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27325e-9fe8-4694-bde5-b6394943f122 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:02.328641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27325e-9fe8-4694-bde5-b6394943f122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:02.338998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27325e-9fe8-4694-bde5-b6394943f122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:02.340583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f27325e-9fe8-4694-bde5-b6394943f122 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:02.351764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 711/1419] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:03.276280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb46e7d-6049-4e50-8070-a5690754fd70 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:03.276903 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb46e7d-6049-4e50-8070-a5690754fd70 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:03.276903 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb46e7d-6049-4e50-8070-a5690754fd70 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:03.278238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:03.284222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebae42e-514b-4380-ada6-c3420ba94c36 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:03.284771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebae42e-514b-4380-ada6-c3420ba94c36 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:03.284771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebae42e-514b-4380-ada6-c3420ba94c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:03.311775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cebae42e-514b-4380-ada6-c3420ba94c36 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:03.312995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cebae42e-514b-4380-ada6-c3420ba94c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:03.321493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:03.473993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb46e7d-6049-4e50-8070-a5690754fd70 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:03.491557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb46e7d-6049-4e50-8070-a5690754fd70 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:03.497219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef720cf-ed18-4ac7-b53d-ec5e584bb4d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:03.497897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef720cf-ed18-4ac7-b53d-ec5e584bb4d8 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:03.498178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef720cf-ed18-4ac7-b53d-ec5e584bb4d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:03.498451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef720cf-ed18-4ac7-b53d-ec5e584bb4d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:03.507646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef720cf-ed18-4ac7-b53d-ec5e584bb4d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:03.509302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef720cf-ed18-4ac7-b53d-ec5e584bb4d8 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:03.519466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 710/1422] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:04.428825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742ba1d-82a3-4b84-bd2e-bb6036c84cce None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:04.429404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742ba1d-82a3-4b84-bd2e-bb6036c84cce None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:04.429404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742ba1d-82a3-4b84-bd2e-bb6036c84cce None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:04.431405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:16:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:04.437155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923598e-764c-44d0-9afa-507b873596de None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:04.437603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923598e-764c-44d0-9afa-507b873596de None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:04.437719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923598e-764c-44d0-9afa-507b873596de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:04.461431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7923598e-764c-44d0-9afa-507b873596de None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:04.462958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7923598e-764c-44d0-9afa-507b873596de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:04.472484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:04.626758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742ba1d-82a3-4b84-bd2e-bb6036c84cce None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:04.650773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742ba1d-82a3-4b84-bd2e-bb6036c84cce None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:04.658628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21189b8-1cbe-4759-bce3-f4566084b836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:04.659307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21189b8-1cbe-4759-bce3-f4566084b836 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:04.659591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21189b8-1cbe-4759-bce3-f4566084b836 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:04.659868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21189b8-1cbe-4759-bce3-f4566084b836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:04.668590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21189b8-1cbe-4759-bce3-f4566084b836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:04.670205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21189b8-1cbe-4759-bce3-f4566084b836 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:04.680303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 714/1425] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:05.587416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e5173-4d6c-4571-ad68-cad94493a3ea None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:05.587999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e5173-4d6c-4571-ad68-cad94493a3ea None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:05.587999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e5173-4d6c-4571-ad68-cad94493a3ea None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:05.589402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:05.594776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320366-65a4-40bf-a418-05c94a5affe1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:05.595222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320366-65a4-40bf-a418-05c94a5affe1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:05.595318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320366-65a4-40bf-a418-05c94a5affe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:05.615763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-62320366-65a4-40bf-a418-05c94a5affe1 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:05.616966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-62320366-65a4-40bf-a418-05c94a5affe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:05.626738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:05.781170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e5173-4d6c-4571-ad68-cad94493a3ea None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:05.802907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e5173-4d6c-4571-ad68-cad94493a3ea None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:05.808983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948f7d-4552-4e76-86ea-4531117c4fa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:05.809724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948f7d-4552-4e76-86ea-4531117c4fa8 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:05.810005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948f7d-4552-4e76-86ea-4531117c4fa8 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:05.810282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948f7d-4552-4e76-86ea-4531117c4fa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:05.819053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948f7d-4552-4e76-86ea-4531117c4fa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:05.820625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948f7d-4552-4e76-86ea-4531117c4fa8 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:05.830863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 713/1428] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:06.675039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9ffe2-9261-4f1d-8997-1e40fde62ad5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:06.675994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9ffe2-9261-4f1d-8997-1e40fde62ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:06.675994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9ffe2-9261-4f1d-8997-1e40fde62ad5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:06.678157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:16:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:06.685798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11a6cd-a2f3-4afe-af19-2bbdd37d740b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:06.686238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11a6cd-a2f3-4afe-af19-2bbdd37d740b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:06.686407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11a6cd-a2f3-4afe-af19-2bbdd37d740b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:06.706480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7c11a6cd-a2f3-4afe-af19-2bbdd37d740b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:06.707657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7c11a6cd-a2f3-4afe-af19-2bbdd37d740b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:06.716323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:16:06.870101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9ffe2-9261-4f1d-8997-1e40fde62ad5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:06.888235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9ffe2-9261-4f1d-8997-1e40fde62ad5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:06.895289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ac7ca-1c9e-49e3-8783-111d68cf1f54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:06.895953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac7ca-1c9e-49e3-8783-111d68cf1f54 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:06.896249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac7ca-1c9e-49e3-8783-111d68cf1f54 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:06.896548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ac7ca-1c9e-49e3-8783-111d68cf1f54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:06.905928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ac7ca-1c9e-49e3-8783-111d68cf1f54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:06.907554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ac7ca-1c9e-49e3-8783-111d68cf1f54 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:06.917737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 717/1431] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:07.785276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a92e4-2b2c-4bc0-ad12-fcc76e5a209d None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:07.785840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a92e4-2b2c-4bc0-ad12-fcc76e5a209d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:07.785840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a92e4-2b2c-4bc0-ad12-fcc76e5a209d None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:07.787125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:16:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:07.792534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40595520-d775-406e-8b29-9204998bc01d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:07.792975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40595520-d775-406e-8b29-9204998bc01d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:07.793163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40595520-d775-406e-8b29-9204998bc01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:07.813992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-40595520-d775-406e-8b29-9204998bc01d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:07.815198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-40595520-d775-406e-8b29-9204998bc01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:07.827863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:16:07.982296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a92e4-2b2c-4bc0-ad12-fcc76e5a209d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:08.004460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a92e4-2b2c-4bc0-ad12-fcc76e5a209d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:08.010386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239976fd-72a6-444a-b73f-46e878909c73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:08.011035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239976fd-72a6-444a-b73f-46e878909c73 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:08.011314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239976fd-72a6-444a-b73f-46e878909c73 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:08.011587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239976fd-72a6-444a-b73f-46e878909c73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:08.020192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239976fd-72a6-444a-b73f-46e878909c73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:08.021855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239976fd-72a6-444a-b73f-46e878909c73 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:08.031430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 716/1434] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:08.901746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869144-13b0-49d0-bf60-b79d092db753 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:08.902309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869144-13b0-49d0-bf60-b79d092db753 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:08.902309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869144-13b0-49d0-bf60-b79d092db753 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:08.903601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:16:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:08.909793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f47f66-793f-4852-afc4-85745348cce6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:08.910554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f47f66-793f-4852-afc4-85745348cce6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:08.910677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f47f66-793f-4852-afc4-85745348cce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:08.935256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f8f47f66-793f-4852-afc4-85745348cce6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:08.936472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f8f47f66-793f-4852-afc4-85745348cce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:08.946018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:09.097415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869144-13b0-49d0-bf60-b79d092db753 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:09.118285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869144-13b0-49d0-bf60-b79d092db753 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:09.124927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69bbc35-505b-4a26-9cfc-dc3f1ad34c8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:09.125623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bbc35-505b-4a26-9cfc-dc3f1ad34c8d service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:09.125909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bbc35-505b-4a26-9cfc-dc3f1ad34c8d service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:09.126178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69bbc35-505b-4a26-9cfc-dc3f1ad34c8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:09.134572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bbc35-505b-4a26-9cfc-dc3f1ad34c8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:09.136144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69bbc35-505b-4a26-9cfc-dc3f1ad34c8d service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:09.145746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 720/1437] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:09.992101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d2ea2-15b3-4022-a084-f7004caa7aef None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:09.992699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d2ea2-15b3-4022-a084-f7004caa7aef None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:09.992699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7d2ea2-15b3-4022-a084-f7004caa7aef None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:09.994075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:10.000215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82be7ea-72e6-461a-a932-4fd441e0d6b8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:10.001023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82be7ea-72e6-461a-a932-4fd441e0d6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:10.001169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82be7ea-72e6-461a-a932-4fd441e0d6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:10.024416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e82be7ea-72e6-461a-a932-4fd441e0d6b8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:10.025641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e82be7ea-72e6-461a-a932-4fd441e0d6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:10.034579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:10.183514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d2ea2-15b3-4022-a084-f7004caa7aef None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:10.199994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7d2ea2-15b3-4022-a084-f7004caa7aef None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:10.206166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5793fa5-499f-4d52-8ae9-30b37392a009 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:10.206813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793fa5-499f-4d52-8ae9-30b37392a009 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:10.207096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793fa5-499f-4d52-8ae9-30b37392a009 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:10.207372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5793fa5-499f-4d52-8ae9-30b37392a009 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:10.215954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5793fa5-499f-4d52-8ae9-30b37392a009 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:10.217594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5793fa5-499f-4d52-8ae9-30b37392a009 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:10.227203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 719/1440] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:12.237606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141e69-81ef-4249-9674-eebf9c4b9587 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:12.238515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141e69-81ef-4249-9674-eebf9c4b9587 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:12.238515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141e69-81ef-4249-9674-eebf9c4b9587 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:12.240861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:12.250034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d91e0-5b94-4bbe-bb9d-858df6735dc2 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:12.250473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d91e0-5b94-4bbe-bb9d-858df6735dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:12.250610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d91e0-5b94-4bbe-bb9d-858df6735dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:12.271047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-887d91e0-5b94-4bbe-bb9d-858df6735dc2 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:12.272271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-887d91e0-5b94-4bbe-bb9d-858df6735dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:12.280867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:12.319847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18141e69-81ef-4249-9674-eebf9c4b9587 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:12.332406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18141e69-81ef-4249-9674-eebf9c4b9587 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:12.336656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4183b31-fcdc-4c3b-b4f2-f56f6b0e8441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:12.337352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4183b31-fcdc-4c3b-b4f2-f56f6b0e8441 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:12.337935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4183b31-fcdc-4c3b-b4f2-f56f6b0e8441 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:12.338215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4183b31-fcdc-4c3b-b4f2-f56f6b0e8441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:12.347818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4183b31-fcdc-4c3b-b4f2-f56f6b0e8441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:12.349356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4183b31-fcdc-4c3b-b4f2-f56f6b0e8441 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:12.361423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 723/1443] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:14.752641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410fe1-de3e-4207-a8e9-e2d80ee767e2 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:14.753224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410fe1-de3e-4207-a8e9-e2d80ee767e2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:14.753224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56410fe1-de3e-4207-a8e9-e2d80ee767e2 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:14.754528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:14.761048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd83461-5a9e-46da-9fe2-19cd74b20574 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:14.761826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd83461-5a9e-46da-9fe2-19cd74b20574 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:14.762000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd83461-5a9e-46da-9fe2-19cd74b20574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:14.782428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bcd83461-5a9e-46da-9fe2-19cd74b20574 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:14.783638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bcd83461-5a9e-46da-9fe2-19cd74b20574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:14.792815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:14.834683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56410fe1-de3e-4207-a8e9-e2d80ee767e2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:14.854831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56410fe1-de3e-4207-a8e9-e2d80ee767e2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:14.861026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e40ae-834b-433f-b2a9-bd22659d094c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:14.861757 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e40ae-834b-433f-b2a9-bd22659d094c service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:14.862034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e40ae-834b-433f-b2a9-bd22659d094c service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:14.862299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e40ae-834b-433f-b2a9-bd22659d094c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:14.872493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e40ae-834b-433f-b2a9-bd22659d094c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:14.873961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e40ae-834b-433f-b2a9-bd22659d094c service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:14.885481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 722/1446] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:17.366956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bac2e-2323-45d4-9626-f721456bb274 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:17.367562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bac2e-2323-45d4-9626-f721456bb274 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:17.367562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0bac2e-2323-45d4-9626-f721456bb274 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:17.368884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:17.374399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb1ff4-15a0-403f-b58a-8e6bbb2fe49f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:17.374948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb1ff4-15a0-403f-b58a-8e6bbb2fe49f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:17.375060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb1ff4-15a0-403f-b58a-8e6bbb2fe49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:17.394128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-83eb1ff4-15a0-403f-b58a-8e6bbb2fe49f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:17.395318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-83eb1ff4-15a0-403f-b58a-8e6bbb2fe49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:17.404054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:17.442540 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bac2e-2323-45d4-9626-f721456bb274 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:17.454496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0bac2e-2323-45d4-9626-f721456bb274 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:17.458719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53f8eb-74b0-4b0e-8cc3-a490046a9a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:17.459357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53f8eb-74b0-4b0e-8cc3-a490046a9a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:17.459634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53f8eb-74b0-4b0e-8cc3-a490046a9a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:17.459908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53f8eb-74b0-4b0e-8cc3-a490046a9a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:17.470117 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53f8eb-74b0-4b0e-8cc3-a490046a9a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:17.471543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53f8eb-74b0-4b0e-8cc3-a490046a9a34 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:17.481969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 726/1449] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:19.768046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b038-d1bc-483a-b672-6c75161f7f8a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:19.768844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b038-d1bc-483a-b672-6c75161f7f8a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:19.768844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db6b038-d1bc-483a-b672-6c75161f7f8a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:19.770584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:19.777946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21d1a-b381-400e-ab23-2e599a861c60 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:19.778424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21d1a-b381-400e-ab23-2e599a861c60 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:19.778537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21d1a-b381-400e-ab23-2e599a861c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:19.800438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8dc21d1a-b381-400e-ab23-2e599a861c60 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:19.802294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8dc21d1a-b381-400e-ab23-2e599a861c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:19.815326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:19.857000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6b038-d1bc-483a-b672-6c75161f7f8a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:19.876412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db6b038-d1bc-483a-b672-6c75161f7f8a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:19.881217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82399853-3031-429b-a0ba-af87e39ebffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:19.882053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399853-3031-429b-a0ba-af87e39ebffe service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:19.882535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399853-3031-429b-a0ba-af87e39ebffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:19.883017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82399853-3031-429b-a0ba-af87e39ebffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:19.904193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82399853-3031-429b-a0ba-af87e39ebffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:19.905680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82399853-3031-429b-a0ba-af87e39ebffe service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:19.920785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 725/1452] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:22.522846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b86279-bb72-41d1-8758-c06c51dc89d9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:22.523359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b86279-bb72-41d1-8758-c06c51dc89d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:22.523359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b86279-bb72-41d1-8758-c06c51dc89d9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:22.524784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:22.530416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89758-bbea-48e3-a56c-9608a7617233 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:22.530875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89758-bbea-48e3-a56c-9608a7617233 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:22.530986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89758-bbea-48e3-a56c-9608a7617233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:22.550595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f3f89758-bbea-48e3-a56c-9608a7617233 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:22.551959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f3f89758-bbea-48e3-a56c-9608a7617233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:22.560941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:16:22.605984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b86279-bb72-41d1-8758-c06c51dc89d9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:22.607880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57b86279-bb72-41d1-8758-c06c51dc89d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:22.623843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b86279-bb72-41d1-8758-c06c51dc89d9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:22.628474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6956c65-f866-4ebe-a0af-a79b26efff5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:22.629142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956c65-f866-4ebe-a0af-a79b26efff5a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:22.629451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956c65-f866-4ebe-a0af-a79b26efff5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:22.629722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6956c65-f866-4ebe-a0af-a79b26efff5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:22.641028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6956c65-f866-4ebe-a0af-a79b26efff5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:22.643288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6956c65-f866-4ebe-a0af-a79b26efff5a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:22.658155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 729/1455] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:25.055169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf170a9-feea-43cc-9092-b555ac80d507 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:25.055725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf170a9-feea-43cc-9092-b555ac80d507 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:25.055725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf170a9-feea-43cc-9092-b555ac80d507 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:25.057806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:25.066119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f267-0bb8-4cf0-929f-35a101154973 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:25.066665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f267-0bb8-4cf0-929f-35a101154973 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:25.066665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b192f267-0bb8-4cf0-929f-35a101154973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:25.084846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b192f267-0bb8-4cf0-929f-35a101154973 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:25.086086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b192f267-0bb8-4cf0-929f-35a101154973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:25.094427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:25.139974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf170a9-feea-43cc-9092-b555ac80d507 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:25.151786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf170a9-feea-43cc-9092-b555ac80d507 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:25.156285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8061215-717e-4e83-9a7e-eca02682b4dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:25.157300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8061215-717e-4e83-9a7e-eca02682b4dc service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:25.157584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8061215-717e-4e83-9a7e-eca02682b4dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:25.157858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8061215-717e-4e83-9a7e-eca02682b4dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:25.174258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8061215-717e-4e83-9a7e-eca02682b4dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:25.176127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8061215-717e-4e83-9a7e-eca02682b4dc service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:25.190775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 728/1458] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:27.487721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61178dae-0112-4fdf-8ca8-7b7c05e756e7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:27.488312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61178dae-0112-4fdf-8ca8-7b7c05e756e7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:27.488312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61178dae-0112-4fdf-8ca8-7b7c05e756e7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:27.489725 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:27.495104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea873-384d-4cc7-b438-fcd1f5ca8a6f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:27.495747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea873-384d-4cc7-b438-fcd1f5ca8a6f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:27.495747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea873-384d-4cc7-b438-fcd1f5ca8a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:27.501976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=70 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:16:27.514700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-605ea873-384d-4cc7-b438-fcd1f5ca8a6f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:27.515904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-605ea873-384d-4cc7-b438-fcd1f5ca8a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:27.524349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:27.561211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61178dae-0112-4fdf-8ca8-7b7c05e756e7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:27.567816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=73 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:16:27.576241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61178dae-0112-4fdf-8ca8-7b7c05e756e7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:27.580391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192c976-6b56-48c1-b261-af69a2a4b5e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:27.581028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192c976-6b56-48c1-b261-af69a2a4b5e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:27.581327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192c976-6b56-48c1-b261-af69a2a4b5e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:27.581602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192c976-6b56-48c1-b261-af69a2a4b5e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:27.591367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192c976-6b56-48c1-b261-af69a2a4b5e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:27.592783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192c976-6b56-48c1-b261-af69a2a4b5e7 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:27.602382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 732/1461] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:30.004676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0710d3-3f4e-4fb8-8cf9-388ff1ff2136 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:30.005362 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0710d3-3f4e-4fb8-8cf9-388ff1ff2136 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:30.005362 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0710d3-3f4e-4fb8-8cf9-388ff1ff2136 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:30.006653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:30.012986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33e12d-1792-4d79-b81d-0b26d5ddd7ef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:30.013500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33e12d-1792-4d79-b81d-0b26d5ddd7ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:30.013614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33e12d-1792-4d79-b81d-0b26d5ddd7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:30.031999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7d33e12d-1792-4d79-b81d-0b26d5ddd7ef None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:30.033626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7d33e12d-1792-4d79-b81d-0b26d5ddd7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:30.042001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:30.082002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0710d3-3f4e-4fb8-8cf9-388ff1ff2136 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:30.100380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0710d3-3f4e-4fb8-8cf9-388ff1ff2136 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:30.107303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591f761-aeca-4f5d-83fc-c0f7c9047c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:30.108427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591f761-aeca-4f5d-83fc-c0f7c9047c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:30.108850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591f761-aeca-4f5d-83fc-c0f7c9047c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:30.109370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591f761-aeca-4f5d-83fc-c0f7c9047c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:30.119414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591f761-aeca-4f5d-83fc-c0f7c9047c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:30.120849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8591f761-aeca-4f5d-83fc-c0f7c9047c84 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:30.131366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 731/1464] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:32.290670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09ae92-9c7a-4beb-89d7-0b721bd18506 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:32.291267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09ae92-9c7a-4beb-89d7-0b721bd18506 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:32.291267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09ae92-9c7a-4beb-89d7-0b721bd18506 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:32.292587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:32.298636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad9c3-0255-4f1b-a43d-d18b8115a3a6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:32.299061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad9c3-0255-4f1b-a43d-d18b8115a3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:32.299197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfad9c3-0255-4f1b-a43d-d18b8115a3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:32.318539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6bfad9c3-0255-4f1b-a43d-d18b8115a3a6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:32.319765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6bfad9c3-0255-4f1b-a43d-d18b8115a3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:32.329429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:32.367095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09ae92-9c7a-4beb-89d7-0b721bd18506 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:32.379631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09ae92-9c7a-4beb-89d7-0b721bd18506 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:32.383577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb0622-8afd-4bc7-855a-520c49b7a596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:32.384240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb0622-8afd-4bc7-855a-520c49b7a596 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:32.384527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb0622-8afd-4bc7-855a-520c49b7a596 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:32.384809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb0622-8afd-4bc7-855a-520c49b7a596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:32.395015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb0622-8afd-4bc7-855a-520c49b7a596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:32.396451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb0622-8afd-4bc7-855a-520c49b7a596 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:32.406090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 735/1467] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:34.668346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c481-98a1-4099-9a12-ffc0e6c9047f None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:34.668907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c481-98a1-4099-9a12-ffc0e6c9047f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:34.668907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2c481-98a1-4099-9a12-ffc0e6c9047f None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:34.670406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:34.676473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508806e-207b-4249-88ab-67aad4b94545 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:34.676943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508806e-207b-4249-88ab-67aad4b94545 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:34.677050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508806e-207b-4249-88ab-67aad4b94545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:34.696191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3508806e-207b-4249-88ab-67aad4b94545 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:34.697464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3508806e-207b-4249-88ab-67aad4b94545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:34.706664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:34.748273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2c481-98a1-4099-9a12-ffc0e6c9047f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:34.760752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2c481-98a1-4099-9a12-ffc0e6c9047f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:34.764784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a020c7d-c537-4d7d-b937-deee176359ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:34.765453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020c7d-c537-4d7d-b937-deee176359ec service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:34.765732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020c7d-c537-4d7d-b937-deee176359ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:34.766005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a020c7d-c537-4d7d-b937-deee176359ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:34.775505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a020c7d-c537-4d7d-b937-deee176359ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:34.776926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a020c7d-c537-4d7d-b937-deee176359ec service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:34.786579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 734/1470] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:44.690567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=17 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:16:44.782327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=17 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:16:54.800272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053101-b366-437d-a17d-1526df40e825 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:54.800272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053101-b366-437d-a17d-1526df40e825 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:54.800272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d053101-b366-437d-a17d-1526df40e825 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:54.801972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 737/1471] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:16:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:54.808108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bba77-9570-43c3-8083-d94beb6f99de None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:54.808633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bba77-9570-43c3-8083-d94beb6f99de None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:54.808633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bba77-9570-43c3-8083-d94beb6f99de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:54.828199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-992bba77-9570-43c3-8083-d94beb6f99de None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:54.829436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-992bba77-9570-43c3-8083-d94beb6f99de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:54.838487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 735/1472] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:54.888099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c8c2d-ac29-4d9a-8cfd-b432fba1ed6e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:54.888390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c8c2d-ac29-4d9a-8cfd-b432fba1ed6e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:54.888723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c8c2d-ac29-4d9a-8cfd-b432fba1ed6e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:54.890185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 738/1473] 10.222.0.66 () {54 vars in 962 bytes} [Thu Jun 16 17:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:54.896241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d14251-358d-4d24-a586-eb1268247b32 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:54.896571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d14251-358d-4d24-a586-eb1268247b32 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:54.896817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d14251-358d-4d24-a586-eb1268247b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:54.915447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-49d14251-358d-4d24-a586-eb1268247b32 None None] MFA Rules not processed for user `db842931b2394e8684012057347080af`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:54.916731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-49d14251-358d-4d24-a586-eb1268247b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:54.924202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 736/1474] 10.222.0.66 () {58 vars in 1059 bytes} [Thu Jun 16 17:16:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:54.932341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c8c2d-ac29-4d9a-8cfd-b432fba1ed6e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:54.945602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c8c2d-ac29-4d9a-8cfd-b432fba1ed6e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:54.950039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbda5aab-0da9-48cd-b4c3-70b1e7f18807 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:54.950695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda5aab-0da9-48cd-b4c3-70b1e7f18807 service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:54.950981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda5aab-0da9-48cd-b4c3-70b1e7f18807 service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:54.951258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbda5aab-0da9-48cd-b4c3-70b1e7f18807 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:54.966968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda5aab-0da9-48cd-b4c3-70b1e7f18807 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:54.968891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbda5aab-0da9-48cd-b4c3-70b1e7f18807 service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:54.980330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 739/1475] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:16:57.292168 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07fa3-3090-4549-ab14-4e383e7782d4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:57.292791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07fa3-3090-4549-ab14-4e383e7782d4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:57.292791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07fa3-3090-4549-ab14-4e383e7782d4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:57.294118 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 737/1476] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:57.300952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ceb93-c75c-423a-945e-7fcdf1d2b9f0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:57.301749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ceb93-c75c-423a-945e-7fcdf1d2b9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:57.301924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ceb93-c75c-423a-945e-7fcdf1d2b9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:57.325402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-905ceb93-c75c-423a-945e-7fcdf1d2b9f0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:57.326619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-905ceb93-c75c-423a-945e-7fcdf1d2b9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:57.339181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 740/1477] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:57.365829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d07fa3-3090-4549-ab14-4e383e7782d4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:57.383035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d07fa3-3090-4549-ab14-4e383e7782d4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:57.389493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae3fd9-eba9-4a53-a35d-45b64ba85220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:57.390163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3fd9-eba9-4a53-a35d-45b64ba85220 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:57.390446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3fd9-eba9-4a53-a35d-45b64ba85220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:57.390724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae3fd9-eba9-4a53-a35d-45b64ba85220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:57.391466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3fd9-eba9-4a53-a35d-45b64ba85220 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:57.394025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-dcae3fd9-eba9-4a53-a35d-45b64ba85220 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:16:57.394025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jun 16 17:16:57.394025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: Jun 16 17:16:57.395268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae3fd9-eba9-4a53-a35d-45b64ba85220 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:57.416611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 738/1478] 10.222.0.66 () {56 vars in 1108 bytes} [Thu Jun 16 17:16:57 2022] GET /identity/v3/projects => generated 2553 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:16:59.581696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cfa12-3b50-4c05-8b36-9e22d34206e1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:59.582721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cfa12-3b50-4c05-8b36-9e22d34206e1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:59.582721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cfa12-3b50-4c05-8b36-9e22d34206e1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:59.585062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 741/1479] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:16:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:16:59.593120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fa044-0e48-4813-aab2-d24ddaa67354 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:59.593606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fa044-0e48-4813-aab2-d24ddaa67354 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:59.593697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fa044-0e48-4813-aab2-d24ddaa67354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:59.613342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b94fa044-0e48-4813-aab2-d24ddaa67354 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:16:59.614615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b94fa044-0e48-4813-aab2-d24ddaa67354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:16:59.623705 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 739/1480] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:16:59.667923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cfa12-3b50-4c05-8b36-9e22d34206e1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:16:59.683738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cfa12-3b50-4c05-8b36-9e22d34206e1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:16:59.690035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b363ef-1be2-47f4-9a4f-9fed3a3ac480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:16:59.690676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b363ef-1be2-47f4-9a4f-9fed3a3ac480 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:16:59.690958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b363ef-1be2-47f4-9a4f-9fed3a3ac480 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:16:59.691235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b363ef-1be2-47f4-9a4f-9fed3a3ac480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:16:59.700083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b363ef-1be2-47f4-9a4f-9fed3a3ac480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:16:59.701747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b363ef-1be2-47f4-9a4f-9fed3a3ac480 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:16:59.712062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 742/1481] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:17:02.052083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124c116-3b49-444c-9738-327b537f007a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:02.052678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124c116-3b49-444c-9738-327b537f007a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:02.052678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124c116-3b49-444c-9738-327b537f007a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:02.054108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 740/1482] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:17:02.063235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae4138-aad8-4a94-8da3-401ad7b4211b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:02.063923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae4138-aad8-4a94-8da3-401ad7b4211b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:02.064119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae4138-aad8-4a94-8da3-401ad7b4211b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:02.089145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-34ae4138-aad8-4a94-8da3-401ad7b4211b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:17:02.090368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-34ae4138-aad8-4a94-8da3-401ad7b4211b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:17:02.099541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 743/1483] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:17:02.140700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124c116-3b49-444c-9738-327b537f007a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:17:02.154231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124c116-3b49-444c-9738-327b537f007a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:17:02.158274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367db0cd-84b8-4567-a6ca-2b07ede68080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:17:02.158959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db0cd-84b8-4567-a6ca-2b07ede68080 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:02.159487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db0cd-84b8-4567-a6ca-2b07ede68080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:02.159487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367db0cd-84b8-4567-a6ca-2b07ede68080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:02.169963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db0cd-84b8-4567-a6ca-2b07ede68080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:17:02.171449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367db0cd-84b8-4567-a6ca-2b07ede68080 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:17:02.181542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 741/1484] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:17:04.400527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7c5c6-345b-4aaa-a591-c9b4475c9e22 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:04.401436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7c5c6-345b-4aaa-a591-c9b4475c9e22 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:04.401436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7c5c6-345b-4aaa-a591-c9b4475c9e22 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:04.402690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 744/1485] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:17:04.411354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1889a-2716-4d0a-94d3-d76a05c5b28f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:04.411983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1889a-2716-4d0a-94d3-d76a05c5b28f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:04.412098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1889a-2716-4d0a-94d3-d76a05c5b28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:04.433321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-76b1889a-2716-4d0a-94d3-d76a05c5b28f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:17:04.434498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-76b1889a-2716-4d0a-94d3-d76a05c5b28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:17:04.443925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 742/1486] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:17:04.487064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7c5c6-345b-4aaa-a591-c9b4475c9e22 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:17:04.501058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7c5c6-345b-4aaa-a591-c9b4475c9e22 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:17:04.506117 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97cabdc-759e-4c76-be75-754edf09778e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:17:04.506813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cabdc-759e-4c76-be75-754edf09778e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:04.507107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cabdc-759e-4c76-be75-754edf09778e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:04.507384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97cabdc-759e-4c76-be75-754edf09778e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:04.519138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cabdc-759e-4c76-be75-754edf09778e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:17:04.520626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97cabdc-759e-4c76-be75-754edf09778e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:17:04.535651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 745/1487] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:17:06.912051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28215a5e-b0fe-486e-b293-cf55bd1a9daa None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:06.912602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28215a5e-b0fe-486e-b293-cf55bd1a9daa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:06.912602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28215a5e-b0fe-486e-b293-cf55bd1a9daa None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:06.913961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 743/1488] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:17:06.919548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bdb61-ec96-4bf1-afae-298d12906643 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:06.920093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bdb61-ec96-4bf1-afae-298d12906643 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:06.920093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bdb61-ec96-4bf1-afae-298d12906643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:06.940572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3e6bdb61-ec96-4bf1-afae-298d12906643 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:17:06.941907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3e6bdb61-ec96-4bf1-afae-298d12906643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:17:06.950984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 746/1489] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:17:06.994799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28215a5e-b0fe-486e-b293-cf55bd1a9daa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:17:07.008461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28215a5e-b0fe-486e-b293-cf55bd1a9daa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:17:07.012622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95d261c-7bb4-4c41-ae7d-c72628dcb3fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:17:07.013307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d261c-7bb4-4c41-ae7d-c72628dcb3fc service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:07.013601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d261c-7bb4-4c41-ae7d-c72628dcb3fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:07.013859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d261c-7bb4-4c41-ae7d-c72628dcb3fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:07.023577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d261c-7bb4-4c41-ae7d-c72628dcb3fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:17:07.025021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95d261c-7bb4-4c41-ae7d-c72628dcb3fc service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:17:07.034777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 744/1490] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:17:09.322691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306156-a933-48be-b0db-82aaf36573d1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:09.323305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306156-a933-48be-b0db-82aaf36573d1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:09.323305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306156-a933-48be-b0db-82aaf36573d1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:09.324633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 747/1491] 10.222.0.66 () {54 vars in 926 bytes} [Thu Jun 16 17:17:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:17:09.331409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af73e38-66ed-47b5-94d4-18fe1215ec0f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:09.332183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af73e38-66ed-47b5-94d4-18fe1215ec0f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:09.332467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af73e38-66ed-47b5-94d4-18fe1215ec0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:09.356902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5af73e38-66ed-47b5-94d4-18fe1215ec0f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:17:09.358152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5af73e38-66ed-47b5-94d4-18fe1215ec0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:17:09.367989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 745/1492] 10.222.0.66 () {58 vars in 1023 bytes} [Thu Jun 16 17:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:17:09.415430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0306156-a933-48be-b0db-82aaf36573d1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:17:09.432438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0306156-a933-48be-b0db-82aaf36573d1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:17:09.439497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18fd81-5be4-44d1-8964-67751fedc197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:17:09.441118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18fd81-5be4-44d1-8964-67751fedc197 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:17:09.441444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18fd81-5be4-44d1-8964-67751fedc197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:17:09.441724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18fd81-5be4-44d1-8964-67751fedc197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:17:09.452171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18fd81-5be4-44d1-8964-67751fedc197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:17:09.453663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18fd81-5be4-44d1-8964-67751fedc197 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:17:09.464036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 748/1493] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:17:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:17:19.351512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=33 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:17:19.459677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=33 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:18:37.809913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f0db-c128-4b43-b006-e3d7ac6f06a5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:37.810496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f0db-c128-4b43-b006-e3d7ac6f06a5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:37.810613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f0db-c128-4b43-b006-e3d7ac6f06a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:37.831827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f7a1f0db-c128-4b43-b006-e3d7ac6f06a5 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:18:37.833257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f7a1f0db-c128-4b43-b006-e3d7ac6f06a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:18:37.842752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 746/1494] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:18:37.854468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5fa6-ae10-4edb-88ec-71223cf9d120 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:37.855072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5fa6-ae10-4edb-88ec-71223cf9d120 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:37.855072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c5fa6-ae10-4edb-88ec-71223cf9d120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:37.876550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-561c5fa6-ae10-4edb-88ec-71223cf9d120 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:18:37.877865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-561c5fa6-ae10-4edb-88ec-71223cf9d120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:18:37.887727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 749/1495] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:18:37.892948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1f0db-c128-4b43-b006-e3d7ac6f06a5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:37.904494 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1f0db-c128-4b43-b006-e3d7ac6f06a5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:37.910532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:37.911333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:37.911710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:37.912651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:37.913505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:18:37.913841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:37.914887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:18:37.914887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jun 16 17:18:37.914887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: Jun 16 17:18:37.915866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:37.931733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 747/1496] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:18:37.944244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c5fa6-ae10-4edb-88ec-71223cf9d120 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:37.954845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c5fa6-ae10-4edb-88ec-71223cf9d120 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:37.960995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:37.961741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:37.962059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:37.962393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:37.963234 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:37.965248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:18:37.965248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jun 16 17:18:37.965248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: Jun 16 17:18:37.966526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:38.017975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 750/1497] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:18:37 2022] POST /identity/v3/projects => generated 379 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:18:38.023740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:38.036454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac7f39-afae-4355-8b44-56370163a5e6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:38.129049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:38.129712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.130002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.130282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.131056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:38.132262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:18:38.132262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jun 16 17:18:38.132262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: Jun 16 17:18:38.133606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:38.162485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 748/1498] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:18:38 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:18:38.166699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:38.178115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99f8c1-5797-4383-9014-8e78077a6a8c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:38.183518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:38.184212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.184503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.184789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.185581 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:38.186822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:18:38.186822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jun 16 17:18:38.186822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: Jun 16 17:18:38.187977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:38.202226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 751/1499] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:18:38.206421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:38.217057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5add3227-7108-4f03-ad4b-42b00eb0cf22 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:38.222312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242eb4b1-55a6-4864-ac79-928ff659e5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:38.222986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242eb4b1-55a6-4864-ac79-928ff659e5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.223271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242eb4b1-55a6-4864-ac79-928ff659e5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.223549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242eb4b1-55a6-4864-ac79-928ff659e5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.224315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242eb4b1-55a6-4864-ac79-928ff659e5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:38.225574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-242eb4b1-55a6-4864-ac79-928ff659e5e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:18:38.225574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jun 16 17:18:38.225574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: Jun 16 17:18:38.226465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242eb4b1-55a6-4864-ac79-928ff659e5e2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:38.244438 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 749/1500] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:18:38.246112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:38.258096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0251ad-d71f-4f32-9be6-7fa5bda325cc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:38.263003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b682b-f22f-47a0-9ab9-7139a03a96b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:38.263942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b682b-f22f-47a0-9ab9-7139a03a96b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.264189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b682b-f22f-47a0-9ab9-7139a03a96b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.264533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b682b-f22f-47a0-9ab9-7139a03a96b2 admin admin] PATH_INFO: `/v3/projects/087b0eee1cdb4a6f8e356b8eda39c46e/users/f404288279224514923f3624e96db3f4/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.287825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b682b-f22f-47a0-9ab9-7139a03a96b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087b0eee1cdb4a6f8e356b8eda39c46e, user_id=f404288279224514923f3624e96db3f4, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:38.288895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-110b682b-f22f-47a0-9ab9-7139a03a96b2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:18:38.288895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jun 16 17:18:38.288895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: Jun 16 17:18:38.290893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b682b-f22f-47a0-9ab9-7139a03a96b2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:38.309308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 752/1501] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:18:38 2022] PUT /identity/v3/projects/087b0eee1cdb4a6f8e356b8eda39c46e/users/f404288279224514923f3624e96db3f4/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:18:38.321435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8640d9-124c-4e1f-9acc-d96663506e09 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.321753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8640d9-124c-4e1f-9acc-d96663506e09 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.322074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8640d9-124c-4e1f-9acc-d96663506e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.360192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6b8640d9-124c-4e1f-9acc-d96663506e09 None None] MFA Rules not processed for user `f404288279224514923f3624e96db3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:18:38.361466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6b8640d9-124c-4e1f-9acc-d96663506e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:18:38.437292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 750/1502] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:18:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:18:38.456902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056be35-362e-413f-9df9-a0f482368643 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.457203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056be35-362e-413f-9df9-a0f482368643 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.457523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056be35-362e-413f-9df9-a0f482368643 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.459101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 753/1503] 10.222.0.66 () {54 vars in 963 bytes} [Thu Jun 16 17:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:18:38.464410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a81eb-9b49-4eeb-8edb-10a0e049093e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.464837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a81eb-9b49-4eeb-8edb-10a0e049093e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.465123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a81eb-9b49-4eeb-8edb-10a0e049093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.485384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d67a81eb-9b49-4eeb-8edb-10a0e049093e None None] MFA Rules not processed for user `d6179dc2bd1d4ff2a04a8362d63e7b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:18:38.486604 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d67a81eb-9b49-4eeb-8edb-10a0e049093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:18:38.538259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:18:38.550934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 751/1504] 10.222.0.66 () {58 vars in 1060 bytes} [Thu Jun 16 17:18:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:18:38.558927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056be35-362e-413f-9df9-a0f482368643 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:38.571296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056be35-362e-413f-9df9-a0f482368643 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:38.575964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d6d8-3c32-4259-ab21-9effcaf84963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:38.577371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d6d8-3c32-4259-ab21-9effcaf84963 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.577371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d6d8-3c32-4259-ab21-9effcaf84963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.577371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d6d8-3c32-4259-ab21-9effcaf84963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.589132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313d6d8-3c32-4259-ab21-9effcaf84963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:38.590651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313d6d8-3c32-4259-ab21-9effcaf84963 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:38.603149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 754/1505] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:18:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:18:38.657817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466db36-abd2-4fc7-9a7b-056b5e7d6e3a None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.658134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466db36-abd2-4fc7-9a7b-056b5e7d6e3a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.658348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466db36-abd2-4fc7-9a7b-056b5e7d6e3a None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.659689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 752/1506] 10.222.0.66 () {52 vars in 829 bytes} [Thu Jun 16 17:18:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:18:38.665907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d6d8-3c32-4259-ab21-9effcaf84963 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:38.742872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d6d8-3c32-4259-ab21-9effcaf84963 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:38.753180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:38.754511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.754764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.755062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] PATH_INFO: `/v3/users/f404288279224514923f3624e96db3f4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.756529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f404288279224514923f3624e96db3f4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:38.757670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:18:38.757670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jun 16 17:18:38.757670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: Jun 16 17:18:38.758890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:38.848268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f404288279224514923f3624e96db3f4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:38.849601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f404288279224514923f3624e96db3f4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:38.850623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f404288279224514923f3624e96db3f4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:38.879736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f404288279224514923f3624e96db3f4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:38.892846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f404288279224514923f3624e96db3f4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:38.903225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:18:38.916985 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 755/1507] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:18:38 2022] DELETE /identity/v3/users/f404288279224514923f3624e96db3f4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:18:38.922551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f99ad8-880c-4dc2-b60b-8e54a90eab53 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:38.923129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f99ad8-880c-4dc2-b60b-8e54a90eab53 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:38.923241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f99ad8-880c-4dc2-b60b-8e54a90eab53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:38.942929 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-55f99ad8-880c-4dc2-b60b-8e54a90eab53 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:18:38.944267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-55f99ad8-880c-4dc2-b60b-8e54a90eab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:18:39.013673 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 753/1508] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:18:39.026219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:39.027926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:18:39.089109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea304e5-8910-45aa-9a79-3ff8816dd25e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:39.092917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b74acd-d206-4c92-83c9-2e7d9a7cad97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:39.093628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b74acd-d206-4c92-83c9-2e7d9a7cad97 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:39.093916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b74acd-d206-4c92-83c9-2e7d9a7cad97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:39.094197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b74acd-d206-4c92-83c9-2e7d9a7cad97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:39.104952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b74acd-d206-4c92-83c9-2e7d9a7cad97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:39.106436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b74acd-d206-4c92-83c9-2e7d9a7cad97 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:39.117562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 756/1509] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:18:39.556648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f99ad8-880c-4dc2-b60b-8e54a90eab53 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:18:39.558378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-55f99ad8-880c-4dc2-b60b-8e54a90eab53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:18:39.572511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f99ad8-880c-4dc2-b60b-8e54a90eab53 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:18:39.578624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:18:39.580031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:18:39.580442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:18:39.580611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] PATH_INFO: `/v3/projects/087b0eee1cdb4a6f8e356b8eda39c46e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:18:39.581869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=087b0eee1cdb4a6f8e356b8eda39c46e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:18:39.588831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:18:39.588831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jun 16 17:18:39.588831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: Jun 16 17:18:39.588831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:18:39.668738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087b0eee1cdb4a6f8e356b8eda39c46e', 'request_id': 'req-e0da0544-0da5-41f1-9284-8c70ad8dcf76'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:39.687252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087b0eee1cdb4a6f8e356b8eda39c46e', 'request_id': 'req-e0da0544-0da5-41f1-9284-8c70ad8dcf76'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:39.703623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087b0eee1cdb4a6f8e356b8eda39c46e', 'request_id': 'req-e0da0544-0da5-41f1-9284-8c70ad8dcf76'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:39.715998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] The token cache is being invalidate because project 087b0eee1cdb4a6f8e356b8eda39c46e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:18:39.716278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:39.716982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e0da0544-0da5-41f1-9284-8c70ad8dcf76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:18:39.718735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 754/1510] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:18:39 2022] DELETE /identity/v3/projects/087b0eee1cdb4a6f8e356b8eda39c46e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:18:49.112253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=29 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:18:49.712739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:23.752456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ea7bf-33a5-4d6d-9ac1-eb0226eb03d3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.753055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ea7bf-33a5-4d6d-9ac1-eb0226eb03d3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.753055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ea7bf-33a5-4d6d-9ac1-eb0226eb03d3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.754356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:23.760308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec0ee-a98f-4c67-94a5-9842038f9d7a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:23.760919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec0ee-a98f-4c67-94a5-9842038f9d7a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:23.760919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afec0ee-a98f-4c67-94a5-9842038f9d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:23.781971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8afec0ee-a98f-4c67-94a5-9842038f9d7a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:23.783209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8afec0ee-a98f-4c67-94a5-9842038f9d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:23.850380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:24.031880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ea7bf-33a5-4d6d-9ac1-eb0226eb03d3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:24.033722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-437ea7bf-33a5-4d6d-9ac1-eb0226eb03d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:24.119045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ea7bf-33a5-4d6d-9ac1-eb0226eb03d3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:24.125804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5018e6a-ceee-4b63-81b3-10ed5da07794 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:24.126648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018e6a-ceee-4b63-81b3-10ed5da07794 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:24.126977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018e6a-ceee-4b63-81b3-10ed5da07794 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:24.127322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5018e6a-ceee-4b63-81b3-10ed5da07794 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:24.139419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5018e6a-ceee-4b63-81b3-10ed5da07794 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:24.141082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5018e6a-ceee-4b63-81b3-10ed5da07794 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:24.152830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 758/1513] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:20:25.069592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402452fd-db8e-4bab-a506-fa076cd84f27 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.070256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402452fd-db8e-4bab-a506-fa076cd84f27 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.070886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402452fd-db8e-4bab-a506-fa076cd84f27 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.072581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:25.078910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c49d2-b849-4a5e-8588-59cfc13120b3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.079423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c49d2-b849-4a5e-8588-59cfc13120b3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.079528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c49d2-b849-4a5e-8588-59cfc13120b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.100377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-891c49d2-b849-4a5e-8588-59cfc13120b3 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:25.102215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-891c49d2-b849-4a5e-8588-59cfc13120b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:25.111999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:25.271408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402452fd-db8e-4bab-a506-fa076cd84f27 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:25.287042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402452fd-db8e-4bab-a506-fa076cd84f27 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:25.292761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0856ae-23ec-4ecd-9e8e-24628366d5b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:25.293463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0856ae-23ec-4ecd-9e8e-24628366d5b8 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.293750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0856ae-23ec-4ecd-9e8e-24628366d5b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.294026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0856ae-23ec-4ecd-9e8e-24628366d5b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.300148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80677-d799-4b8c-8f41-373c4e14d687 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.300521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80677-d799-4b8c-8f41-373c4e14d687 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.300818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe80677-d799-4b8c-8f41-373c4e14d687 None None] PATH_INFO: `/v3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.302362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 760/1516] 10.222.0.81 () {54 vars in 925 bytes} [Thu Jun 16 17:20:25 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:20:25.303377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0856ae-23ec-4ecd-9e8e-24628366d5b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:25.305307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0856ae-23ec-4ecd-9e8e-24628366d5b8 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:25.309672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b15bd-80e6-4cac-a983-6f41b5e98a29 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:25.309965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b15bd-80e6-4cac-a983-6f41b5e98a29 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:25.310246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31b15bd-80e6-4cac-a983-6f41b5e98a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:25.316370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 757/1517] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:20:25.329864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b31b15bd-80e6-4cac-a983-6f41b5e98a29 None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:25.331352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b31b15bd-80e6-4cac-a983-6f41b5e98a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:25.427345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 761/1518] 10.222.0.81 () {58 vars in 1015 bytes} [Thu Jun 16 17:20:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:29.183944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a3971-6d64-4eff-9d4f-41506056455e None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:29.184663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a3971-6d64-4eff-9d4f-41506056455e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:29.184663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a3971-6d64-4eff-9d4f-41506056455e None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.186401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 758/1519] 10.222.0.66 () {54 vars in 960 bytes} [Thu Jun 16 17:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:29.192227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a442832-1304-4604-8f3a-3c91ca538438 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:29.192755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a442832-1304-4604-8f3a-3c91ca538438 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:29.192865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a442832-1304-4604-8f3a-3c91ca538438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.216802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0a442832-1304-4604-8f3a-3c91ca538438 None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:29.217550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a442832-1304-4604-8f3a-3c91ca538438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:29.227217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 762/1520] 10.222.0.66 () {58 vars in 1057 bytes} [Thu Jun 16 17:20:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:29.235384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a3971-6d64-4eff-9d4f-41506056455e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:29.250055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a3971-6d64-4eff-9d4f-41506056455e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:29.255238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd51583-b4f4-4e33-baec-f59e580000f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:29.255891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd51583-b4f4-4e33-baec-f59e580000f0 service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:29.256170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd51583-b4f4-4e33-baec-f59e580000f0 service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:29.256445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd51583-b4f4-4e33-baec-f59e580000f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:29.266503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd51583-b4f4-4e33-baec-f59e580000f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:29.268106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd51583-b4f4-4e33-baec-f59e580000f0 service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:29.279812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 759/1521] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:20:36.606716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c626684-0264-48e2-924b-44b2737f0fd6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.607314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c626684-0264-48e2-924b-44b2737f0fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.607314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c626684-0264-48e2-924b-44b2737f0fd6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.608634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:36.614787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7984f6-8430-44fd-8dcd-55886de23e22 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.615292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7984f6-8430-44fd-8dcd-55886de23e22 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.615401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7984f6-8430-44fd-8dcd-55886de23e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.635029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8f7984f6-8430-44fd-8dcd-55886de23e22 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:36.636277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8f7984f6-8430-44fd-8dcd-55886de23e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:36.645703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:36.798380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c626684-0264-48e2-924b-44b2737f0fd6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:36.814659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c626684-0264-48e2-924b-44b2737f0fd6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:36.821916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78315aa9-3908-4045-b672-4e05e749e320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:36.822575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78315aa9-3908-4045-b672-4e05e749e320 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:36.822863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78315aa9-3908-4045-b672-4e05e749e320 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:36.823120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78315aa9-3908-4045-b672-4e05e749e320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:36.832317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78315aa9-3908-4045-b672-4e05e749e320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:36.834002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78315aa9-3908-4045-b672-4e05e749e320 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:36.845154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 764/1524] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:20:37.747016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca55b-61b7-48cb-92fc-30301687e603 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.747558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca55b-61b7-48cb-92fc-30301687e603 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.747558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ca55b-61b7-48cb-92fc-30301687e603 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.748918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 17:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:37.754421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6c673-d5b0-46a5-88d2-cf1c4a4dc4a2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.754970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6c673-d5b0-46a5-88d2-cf1c4a4dc4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.754970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c6c673-d5b0-46a5-88d2-cf1c4a4dc4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.776029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a2c6c673-d5b0-46a5-88d2-cf1c4a4dc4a2 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:37.777302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a2c6c673-d5b0-46a5-88d2-cf1c4a4dc4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:37.786040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 17:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:37.947095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ca55b-61b7-48cb-92fc-30301687e603 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:37.963558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ca55b-61b7-48cb-92fc-30301687e603 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:37.970456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d4df8-b545-4932-9d6a-06b14bae5eb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:37.971107 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d4df8-b545-4932-9d6a-06b14bae5eb7 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:37.971417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d4df8-b545-4932-9d6a-06b14bae5eb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:37.971693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d4df8-b545-4932-9d6a-06b14bae5eb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:37.986627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d4df8-b545-4932-9d6a-06b14bae5eb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:37.988346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d4df8-b545-4932-9d6a-06b14bae5eb7 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:37.999908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 762/1527] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:20:42.127596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8da7-de50-468e-bee0-a86728c3056a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.128455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8da7-de50-468e-bee0-a86728c3056a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.128455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a8da7-de50-468e-bee0-a86728c3056a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.129628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:42.135077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda90d-b2ca-454d-8ad6-bc8182461c23 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.135589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda90d-b2ca-454d-8ad6-bc8182461c23 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.135589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eda90d-b2ca-454d-8ad6-bc8182461c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.166965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c0eda90d-b2ca-454d-8ad6-bc8182461c23 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:42.168193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c0eda90d-b2ca-454d-8ad6-bc8182461c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:42.176918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:42.213661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a8da7-de50-468e-bee0-a86728c3056a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:42.215539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-805a8da7-de50-468e-bee0-a86728c3056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:42.283739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:42.289327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a8da7-de50-468e-bee0-a86728c3056a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:42.294625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f8627-948b-4535-864d-6ee3a67ed145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:42.295710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f8627-948b-4535-864d-6ee3a67ed145 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:42.296357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f8627-948b-4535-864d-6ee3a67ed145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:42.296835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f8627-948b-4535-864d-6ee3a67ed145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:42.310375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f8627-948b-4535-864d-6ee3a67ed145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:42.311817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f8627-948b-4535-864d-6ee3a67ed145 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:42.322922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 767/1530] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:20:44.442065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656621-061a-41b8-8425-98da4beb65b1 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.442631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656621-061a-41b8-8425-98da4beb65b1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.442631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9656621-061a-41b8-8425-98da4beb65b1 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.444175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:44.449924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cff8a-dfa0-4ad5-a4c1-6273e96ac685 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.450355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cff8a-dfa0-4ad5-a4c1-6273e96ac685 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.450477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53cff8a-dfa0-4ad5-a4c1-6273e96ac685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.470765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b53cff8a-dfa0-4ad5-a4c1-6273e96ac685 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:44.471976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b53cff8a-dfa0-4ad5-a4c1-6273e96ac685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:44.481586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:44.516968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9656621-061a-41b8-8425-98da4beb65b1 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:44.529755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9656621-061a-41b8-8425-98da4beb65b1 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:44.533798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5de25-cca3-4d54-b91b-94b434ae1b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:44.534463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5de25-cca3-4d54-b91b-94b434ae1b41 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:44.534745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5de25-cca3-4d54-b91b-94b434ae1b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:44.535021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5de25-cca3-4d54-b91b-94b434ae1b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:44.544484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5de25-cca3-4d54-b91b-94b434ae1b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:44.546439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5de25-cca3-4d54-b91b-94b434ae1b41 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:44.555513 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 765/1533] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:20:48.065790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27638ae-da1a-44bf-ae01-f5591f505ea1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.066369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27638ae-da1a-44bf-ae01-f5591f505ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.066369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27638ae-da1a-44bf-ae01-f5591f505ea1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.067771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:48.073646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83b4f-4e2e-4b8d-a3e2-d57fd4026982 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.074124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83b4f-4e2e-4b8d-a3e2-d57fd4026982 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.074251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83b4f-4e2e-4b8d-a3e2-d57fd4026982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.092395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-46d83b4f-4e2e-4b8d-a3e2-d57fd4026982 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:48.093636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-46d83b4f-4e2e-4b8d-a3e2-d57fd4026982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:48.102315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:48.367047 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27638ae-da1a-44bf-ae01-f5591f505ea1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:48.369143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c27638ae-da1a-44bf-ae01-f5591f505ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:48.431899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27638ae-da1a-44bf-ae01-f5591f505ea1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:48.435735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26b551-586a-4a2c-9f6d-be5e2093ee01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:48.436403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26b551-586a-4a2c-9f6d-be5e2093ee01 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:48.436688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26b551-586a-4a2c-9f6d-be5e2093ee01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:48.436965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26b551-586a-4a2c-9f6d-be5e2093ee01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:48.447690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26b551-586a-4a2c-9f6d-be5e2093ee01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:48.449132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26b551-586a-4a2c-9f6d-be5e2093ee01 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:48.459113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 770/1536] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:20:50.586803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055e204-ac2e-4b0e-997c-14227d4e8a30 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.587382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055e204-ac2e-4b0e-997c-14227d4e8a30 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.587382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055e204-ac2e-4b0e-997c-14227d4e8a30 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.588637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 17:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:20:50.594115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb3d0b-dc94-4cc4-8c5e-8c93da58029e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.594556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb3d0b-dc94-4cc4-8c5e-8c93da58029e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.594679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb3d0b-dc94-4cc4-8c5e-8c93da58029e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.615129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-04fb3d0b-dc94-4cc4-8c5e-8c93da58029e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:20:50.616354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-04fb3d0b-dc94-4cc4-8c5e-8c93da58029e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:50.626108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 17:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:20:50.888364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055e204-ac2e-4b0e-997c-14227d4e8a30 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:20:50.892258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4055e204-ac2e-4b0e-997c-14227d4e8a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:20:50.900970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=61 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:20:50.908089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055e204-ac2e-4b0e-997c-14227d4e8a30 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:20:50.911790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f4db2-8750-4e2d-8309-330d73277b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:20:50.912435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4db2-8750-4e2d-8309-330d73277b63 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:20:50.912719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4db2-8750-4e2d-8309-330d73277b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:20:50.912997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f4db2-8750-4e2d-8309-330d73277b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:20:50.922884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f4db2-8750-4e2d-8309-330d73277b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:20:50.924320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f4db2-8750-4e2d-8309-330d73277b63 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:20:50.934080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 768/1539] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:21:00.609400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=36 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:21:00.929752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:19.588864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe4fd9-9839-490a-859a-e53ac6b99791 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.589571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe4fd9-9839-490a-859a-e53ac6b99791 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.589571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfe4fd9-9839-490a-859a-e53ac6b99791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.593161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25d89b-fa4a-4816-b116-e3c464bc8287 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.594265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25d89b-fa4a-4816-b116-e3c464bc8287 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.594770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25d89b-fa4a-4816-b116-e3c464bc8287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.618177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0e25d89b-fa4a-4816-b116-e3c464bc8287 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.619796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0e25d89b-fa4a-4816-b116-e3c464bc8287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.620093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0dfe4fd9-9839-490a-859a-e53ac6b99791 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.621649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0dfe4fd9-9839-490a-859a-e53ac6b99791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.637405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 772/1540] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:19.638077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 769/1541] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:19.639379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b518e-d0b1-42a6-b1c2-b9e54d47a259 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.639669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b518e-d0b1-42a6-b1c2-b9e54d47a259 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.639960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b518e-d0b1-42a6-b1c2-b9e54d47a259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.660399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc2d7b-929c-447f-95ea-f7d2f14c3f01 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.661087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc2d7b-929c-447f-95ea-f7d2f14c3f01 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.661607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc2d7b-929c-447f-95ea-f7d2f14c3f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.682528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-938b518e-d0b1-42a6-b1c2-b9e54d47a259 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.683756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-938b518e-d0b1-42a6-b1c2-b9e54d47a259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.692975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-eacc2d7b-929c-447f-95ea-f7d2f14c3f01 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.694193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-eacc2d7b-929c-447f-95ea-f7d2f14c3f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.714308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 773/1542] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:19.716098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d6d6a-9596-4cee-a143-b08a6b47da95 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.716621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d6d6a-9596-4cee-a143-b08a6b47da95 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.717252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d6d6a-9596-4cee-a143-b08a6b47da95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.718987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 770/1543] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:22:19.731298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e346c-d603-4a17-a9f8-1ef3530377f0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.731536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e346c-d603-4a17-a9f8-1ef3530377f0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.731763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e346c-d603-4a17-a9f8-1ef3530377f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.754762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eb7d6d6a-9596-4cee-a143-b08a6b47da95 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.756453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb7d6d6a-9596-4cee-a143-b08a6b47da95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.770587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c77e346c-d603-4a17-a9f8-1ef3530377f0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.771837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c77e346c-d603-4a17-a9f8-1ef3530377f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.773044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 774/1544] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:19.774208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d6d6a-9596-4cee-a143-b08a6b47da95 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:19.792633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d6d6a-9596-4cee-a143-b08a6b47da95 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:19.801690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 771/1545] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:19.802637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:19.803722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e346c-d603-4a17-a9f8-1ef3530377f0 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:19.804631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.809710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.810367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.812019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:19.816419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:19.817760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:22:19.817760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jun 16 17:22:19.817760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: Jun 16 17:22:19.819070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:19.832245 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e346c-d603-4a17-a9f8-1ef3530377f0 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:19.842051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:19.844393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 775/1546] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:19.845353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.845865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.846368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.847313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:19.848387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:19.848926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:19.856515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:19.871235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d888b10-8623-4e6b-bc26-9d9cfc5ad552 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:19.879033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 772/1547] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:19.880559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:19.885645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ee933-06c5-4392-8970-0241dcf22e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:19.886346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ee933-06c5-4392-8970-0241dcf22e16 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.886637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ee933-06c5-4392-8970-0241dcf22e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.886923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ee933-06c5-4392-8970-0241dcf22e16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.887941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ee933-06c5-4392-8970-0241dcf22e16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:19.888260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ee933-06c5-4392-8970-0241dcf22e16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:19.889914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ee933-06c5-4392-8970-0241dcf22e16 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:19.896721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1616134-3ddc-4491-9c42-09651047f0a7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:19.901995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d215f0-09e1-486e-83a3-c66087969208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:19.902718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d215f0-09e1-486e-83a3-c66087969208 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.903010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d215f0-09e1-486e-83a3-c66087969208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.903293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d215f0-09e1-486e-83a3-c66087969208 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.903994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d215f0-09e1-486e-83a3-c66087969208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:19.905554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 776/1548] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:19.907136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ac0e3-7374-40a4-b27a-76b3901916d8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.907582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-b0d215f0-09e1-486e-83a3-c66087969208 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:22:19.907582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jun 16 17:22:19.907582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: Jun 16 17:22:19.908047 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ac0e3-7374-40a4-b27a-76b3901916d8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.908532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ac0e3-7374-40a4-b27a-76b3901916d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.908786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d215f0-09e1-486e-83a3-c66087969208 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:19.928602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bc2ac0e3-7374-40a4-b27a-76b3901916d8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.930055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bc2ac0e3-7374-40a4-b27a-76b3901916d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.938233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 777/1549] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:19.938431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 773/1550] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:19.941257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99849550-1e55-4271-8085-c324bc07af2d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.941257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99849550-1e55-4271-8085-c324bc07af2d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.941257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99849550-1e55-4271-8085-c324bc07af2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.947134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10fc1d-36ec-4caa-944b-0ef73dbeb7b3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:19.948004 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10fc1d-36ec-4caa-944b-0ef73dbeb7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:19.948512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10fc1d-36ec-4caa-944b-0ef73dbeb7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:19.960435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-99849550-1e55-4271-8085-c324bc07af2d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.961984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99849550-1e55-4271-8085-c324bc07af2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:19.969359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-af10fc1d-36ec-4caa-944b-0ef73dbeb7b3 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:19.970534 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-af10fc1d-36ec-4caa-944b-0ef73dbeb7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.021435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 778/1551] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.022152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99849550-1e55-4271-8085-c324bc07af2d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.031984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99849550-1e55-4271-8085-c324bc07af2d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.036516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2d32d-16cb-41b7-a4fa-f9ec8e691efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.037226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d32d-16cb-41b7-a4fa-f9ec8e691efd admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.037478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d32d-16cb-41b7-a4fa-f9ec8e691efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.037756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2d32d-16cb-41b7-a4fa-f9ec8e691efd admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.038468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2d32d-16cb-41b7-a4fa-f9ec8e691efd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.039224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 774/1552] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.040166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10fc1d-36ec-4caa-944b-0ef73dbeb7b3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.040702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2d32d-16cb-41b7-a4fa-f9ec8e691efd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.052574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10fc1d-36ec-4caa-944b-0ef73dbeb7b3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.058325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33964c-33da-4031-84b8-5306b02df75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.059115 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33964c-33da-4031-84b8-5306b02df75c admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.059395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33964c-33da-4031-84b8-5306b02df75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.059805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33964c-33da-4031-84b8-5306b02df75c admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.060620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33964c-33da-4031-84b8-5306b02df75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.062565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33964c-33da-4031-84b8-5306b02df75c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.071216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 779/1553] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.071216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af872baf-f069-4077-b8d3-7f24e97d6322 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.071216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af872baf-f069-4077-b8d3-7f24e97d6322 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.071216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af872baf-f069-4077-b8d3-7f24e97d6322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.090139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-af872baf-f069-4077-b8d3-7f24e97d6322 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:20.094378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 775/1554] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.094538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-af872baf-f069-4077-b8d3-7f24e97d6322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.094636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33964c-33da-4031-84b8-5306b02df75c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.151338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 780/1555] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.153902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33964c-33da-4031-84b8-5306b02df75c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.158841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaa40-cee0-496c-9f04-2800bae3ea73 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.159256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaa40-cee0-496c-9f04-2800bae3ea73 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.159635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95aaa40-cee0-496c-9f04-2800bae3ea73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.160335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db248d3c-adaa-4d3f-98e0-9ce7cd67c812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.161145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db248d3c-adaa-4d3f-98e0-9ce7cd67c812 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.161460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db248d3c-adaa-4d3f-98e0-9ce7cd67c812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.161741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db248d3c-adaa-4d3f-98e0-9ce7cd67c812 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.162474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db248d3c-adaa-4d3f-98e0-9ce7cd67c812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.164914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db248d3c-adaa-4d3f-98e0-9ce7cd67c812 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.179314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e95aaa40-cee0-496c-9f04-2800bae3ea73 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:20.180982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e95aaa40-cee0-496c-9f04-2800bae3ea73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.188172 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 776/1556] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.189284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0da5a6-be80-4c4f-b33c-240bbdf42a56 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.189576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0da5a6-be80-4c4f-b33c-240bbdf42a56 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.189858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0da5a6-be80-4c4f-b33c-240bbdf42a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.189984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 781/1557] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.191238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813589b-0bff-41ec-878f-65a82de82fd9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.191632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813589b-0bff-41ec-878f-65a82de82fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.191954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813589b-0bff-41ec-878f-65a82de82fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.211125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4813589b-0bff-41ec-878f-65a82de82fd9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:20.211527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6c0da5a6-be80-4c4f-b33c-240bbdf42a56 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:20.212387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6c0da5a6-be80-4c4f-b33c-240bbdf42a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.212506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4813589b-0bff-41ec-878f-65a82de82fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.220460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 777/1558] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.220990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 782/1559] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.221696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c332d-f2ad-4f41-a8d8-3194348597a4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.221980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c332d-f2ad-4f41-a8d8-3194348597a4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.222240 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c332d-f2ad-4f41-a8d8-3194348597a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.224987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813589b-0bff-41ec-878f-65a82de82fd9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.235566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813589b-0bff-41ec-878f-65a82de82fd9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.237280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:20.241488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faa5432-61bc-4e9b-8aa2-36295bc6a623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.242254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa5432-61bc-4e9b-8aa2-36295bc6a623 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.242563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa5432-61bc-4e9b-8aa2-36295bc6a623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.242944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faa5432-61bc-4e9b-8aa2-36295bc6a623 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.244104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa5432-61bc-4e9b-8aa2-36295bc6a623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.244989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-4faa5432-61bc-4e9b-8aa2-36295bc6a623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:22:20.244989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jun 16 17:22:20.244989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: Jun 16 17:22:20.246590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faa5432-61bc-4e9b-8aa2-36295bc6a623 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.253370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-047c332d-f2ad-4f41-a8d8-3194348597a4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:20.254763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-047c332d-f2ad-4f41-a8d8-3194348597a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.258997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:20.264574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 778/1560] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.265568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c332d-f2ad-4f41-a8d8-3194348597a4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.271828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 783/1561] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/users => generated 297 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.273030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63ec4a-1bbd-4e21-b7e8-0a648ae05dfd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.273581 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63ec4a-1bbd-4e21-b7e8-0a648ae05dfd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.273818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63ec4a-1bbd-4e21-b7e8-0a648ae05dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.276509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c332d-f2ad-4f41-a8d8-3194348597a4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.281466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e997d1-d8ce-411f-ae4a-7542efd858e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.282174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e997d1-d8ce-411f-ae4a-7542efd858e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.282453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e997d1-d8ce-411f-ae4a-7542efd858e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.282729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e997d1-d8ce-411f-ae4a-7542efd858e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.283457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e997d1-d8ce-411f-ae4a-7542efd858e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.285430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e997d1-d8ce-411f-ae4a-7542efd858e5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.293872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ae63ec4a-1bbd-4e21-b7e8-0a648ae05dfd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:20.295441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ae63ec4a-1bbd-4e21-b7e8-0a648ae05dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.304667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 784/1562] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.306266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 779/1563] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/users => generated 282 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.307064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e997d1-d8ce-411f-ae4a-7542efd858e5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.309375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63ec4a-1bbd-4e21-b7e8-0a648ae05dfd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.316945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e997d1-d8ce-411f-ae4a-7542efd858e5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.319160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63ec4a-1bbd-4e21-b7e8-0a648ae05dfd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.322967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.323656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.323935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.324209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.324859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f16f9-f2c7-42aa-b7ec-e00aa8420554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.324975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:20.325231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.325748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f16f9-f2c7-42aa-b7ec-e00aa8420554 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.326104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f16f9-f2c7-42aa-b7ec-e00aa8420554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.326452 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f16f9-f2c7-42aa-b7ec-e00aa8420554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.326912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.327329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f16f9-f2c7-42aa-b7ec-e00aa8420554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.328988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f16f9-f2c7-42aa-b7ec-e00aa8420554 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.339813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 780/1564] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:20.340700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.343049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 785/1565] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:20.344324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d8b2-a30c-4b27-aa6b-17b84ad0ece8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.344626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d8b2-a30c-4b27-aa6b-17b84ad0ece8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.344910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8d8b2-a30c-4b27-aa6b-17b84ad0ece8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.352062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276375a-28b7-4563-b5a4-5495592fce52 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.364222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2cd8d8b2-a30c-4b27-aa6b-17b84ad0ece8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:20.365473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2cd8d8b2-a30c-4b27-aa6b-17b84ad0ece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:20.365633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.366555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.371523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.371980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.372714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:20.373037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.374707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.375671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 786/1566] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:20.376604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8d8b2-a30c-4b27-aa6b-17b84ad0ece8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.388160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8d8b2-a30c-4b27-aa6b-17b84ad0ece8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.389348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 781/1567] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:20.390430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.394001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.394676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.394962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.395242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.396025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:20.396296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.398665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.405337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cefc26-0332-4b8c-a021-76774563be27 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.415851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 787/1568] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:20.416082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6445f0-e2bb-4d44-9480-d8ed54c165cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.416197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.416278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6445f0-e2bb-4d44-9480-d8ed54c165cf admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.416278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6445f0-e2bb-4d44-9480-d8ed54c165cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.416278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6445f0-e2bb-4d44-9480-d8ed54c165cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.416278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6445f0-e2bb-4d44-9480-d8ed54c165cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.416278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6445f0-e2bb-4d44-9480-d8ed54c165cf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.424023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66af2ec-28ed-4b8b-bddf-2bff41275228 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.427453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 782/1569] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:20.429928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.430890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.431397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.431570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6445f0-e2bb-4d44-9480-d8ed54c165cf admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.431728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.432431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:20.432738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.434383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.446628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6445f0-e2bb-4d44-9480-d8ed54c165cf admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.448370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9a4bc-45dd-4177-9d79-be126c80012f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.449067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9a4bc-45dd-4177-9d79-be126c80012f admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.449379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9a4bc-45dd-4177-9d79-be126c80012f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.449656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9a4bc-45dd-4177-9d79-be126c80012f admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.450393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9a4bc-45dd-4177-9d79-be126c80012f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.452023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9a4bc-45dd-4177-9d79-be126c80012f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.455959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 788/1570] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:20.461437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.468144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 783/1571] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:20.469526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9a4bc-45dd-4177-9d79-be126c80012f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.473622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132529b-e20c-4c80-b544-b83b6f894949 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.481487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462465ff-0deb-4971-901d-3994cff16b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.482179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462465ff-0deb-4971-901d-3994cff16b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.482465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462465ff-0deb-4971-901d-3994cff16b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.482745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462465ff-0deb-4971-901d-3994cff16b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.483489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462465ff-0deb-4971-901d-3994cff16b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.485148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462465ff-0deb-4971-901d-3994cff16b2e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.485952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9a4bc-45dd-4177-9d79-be126c80012f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.493638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889ae47-fdc8-412e-a5e2-65f8e09ed35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.494318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889ae47-fdc8-412e-a5e2-65f8e09ed35c admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.494603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889ae47-fdc8-412e-a5e2-65f8e09ed35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.494875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889ae47-fdc8-412e-a5e2-65f8e09ed35c admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.495572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889ae47-fdc8-412e-a5e2-65f8e09ed35c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.496405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 789/1572] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:20.497297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462465ff-0deb-4971-901d-3994cff16b2e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.500832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889ae47-fdc8-412e-a5e2-65f8e09ed35c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.507873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462465ff-0deb-4971-901d-3994cff16b2e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.512881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.513597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.513914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.514209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.514884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:20.515221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.516834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.530378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 790/1573] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:20.531109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.537861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 784/1574] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.537861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889ae47-fdc8-412e-a5e2-65f8e09ed35c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.543715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb7c5c-d89d-42a2-9e14-8c101a45012b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.609564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889ae47-fdc8-412e-a5e2-65f8e09ed35c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.614909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe5fe3-a5f2-4579-8822-c8040847256c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.615762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe5fe3-a5f2-4579-8822-c8040847256c admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.616114 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aab2dc-b92f-43de-b688-4633ffd7d510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.616536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe5fe3-a5f2-4579-8822-c8040847256c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.616536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe5fe3-a5f2-4579-8822-c8040847256c admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.616781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aab2dc-b92f-43de-b688-4633ffd7d510 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.617110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aab2dc-b92f-43de-b688-4633ffd7d510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.617492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aab2dc-b92f-43de-b688-4633ffd7d510 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.617824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe5fe3-a5f2-4579-8822-c8040847256c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.618405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aab2dc-b92f-43de-b688-4633ffd7d510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.619893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe5fe3-a5f2-4579-8822-c8040847256c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.620316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aab2dc-b92f-43de-b688-4633ffd7d510 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.651084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 791/1575] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/projects => generated 403 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.651084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aab2dc-b92f-43de-b688-4633ffd7d510 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.653092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 785/1576] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.653991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe5fe3-a5f2-4579-8822-c8040847256c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.718154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aab2dc-b92f-43de-b688-4633ffd7d510 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.720955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:20.724299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ff398-8a08-422b-894a-0d3f72573e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.724997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff398-8a08-422b-894a-0d3f72573e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.725456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff398-8a08-422b-894a-0d3f72573e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.725936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff398-8a08-422b-894a-0d3f72573e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.726494 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe5fe3-a5f2-4579-8822-c8040847256c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.727603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ff398-8a08-422b-894a-0d3f72573e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.730516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ff398-8a08-422b-894a-0d3f72573e5e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.732995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83967e2-e13b-4574-a1d3-76429e93787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.733695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83967e2-e13b-4574-a1d3-76429e93787e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.733976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83967e2-e13b-4574-a1d3-76429e93787e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.734347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83967e2-e13b-4574-a1d3-76429e93787e admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.735040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83967e2-e13b-4574-a1d3-76429e93787e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.736921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83967e2-e13b-4574-a1d3-76429e93787e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.749394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 792/1577] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:20.749394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ff398-8a08-422b-894a-0d3f72573e5e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.756702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:20.764820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ff398-8a08-422b-894a-0d3f72573e5e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.767838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 786/1578] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.768780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83967e2-e13b-4574-a1d3-76429e93787e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.770595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cd745-2738-449f-a49a-97f94a0d0fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.771272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cd745-2738-449f-a49a-97f94a0d0fef admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.771563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cd745-2738-449f-a49a-97f94a0d0fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.772075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76cd745-2738-449f-a49a-97f94a0d0fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.773292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cd745-2738-449f-a49a-97f94a0d0fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.775348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76cd745-2738-449f-a49a-97f94a0d0fef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.792620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 793/1579] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:20.794028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cd745-2738-449f-a49a-97f94a0d0fef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.806508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76cd745-2738-449f-a49a-97f94a0d0fef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.837288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83967e2-e13b-4574-a1d3-76429e93787e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.842144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.843046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.843323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.843593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] PATH_INFO: `/v3/projects/859da351f9cc437ebe79e5bdb87a177d/users/a132fd2fe24343239eafa8c46abaa2ed/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.864664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=859da351f9cc437ebe79e5bdb87a177d, user_id=a132fd2fe24343239eafa8c46abaa2ed, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.865781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:22:20.865781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jun 16 17:22:20.865781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: Jun 16 17:22:20.867916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.873063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c8ed1-8af7-451c-8f25-46fcf14b4775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.873915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c8ed1-8af7-451c-8f25-46fcf14b4775 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.874332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c8ed1-8af7-451c-8f25-46fcf14b4775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.874726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c8ed1-8af7-451c-8f25-46fcf14b4775 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.875589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c8ed1-8af7-451c-8f25-46fcf14b4775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.877606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c8ed1-8af7-451c-8f25-46fcf14b4775 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.882838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 787/1580] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:20 2022] PUT /identity/v3/projects/859da351f9cc437ebe79e5bdb87a177d/users/a132fd2fe24343239eafa8c46abaa2ed/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:20.887094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.901384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 794/1581] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.902201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c8ed1-8af7-451c-8f25-46fcf14b4775 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:20.916381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c8ed1-8af7-451c-8f25-46fcf14b4775 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.955124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38eb36b-ea0c-4058-bd89-e3431bbb3f51 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:20.960128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43fe01-b2c0-4f7f-8fd5-59a1c06ff2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.960857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43fe01-b2c0-4f7f-8fd5-59a1c06ff2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.961204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43fe01-b2c0-4f7f-8fd5-59a1c06ff2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.961521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb43fe01-b2c0-4f7f-8fd5-59a1c06ff2e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.962225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43fe01-b2c0-4f7f-8fd5-59a1c06ff2e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.964260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb43fe01-b2c0-4f7f-8fd5-59a1c06ff2e4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.981791 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc07a2-f65e-4a23-84d4-d954966c81c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:20.983764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc07a2-f65e-4a23-84d4-d954966c81c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:20.983764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc07a2-f65e-4a23-84d4-d954966c81c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:20.983764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc07a2-f65e-4a23-84d4-d954966c81c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:20.983764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc07a2-f65e-4a23-84d4-d954966c81c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:20.985743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dc07a2-f65e-4a23-84d4-d954966c81c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:20.993759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 788/1582] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:20.994680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43fe01-b2c0-4f7f-8fd5-59a1c06ff2e4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.008595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 795/1583] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:21.009541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc07a2-f65e-4a23-84d4-d954966c81c9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.020536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc07a2-f65e-4a23-84d4-d954966c81c9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.063896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:21.077249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb43fe01-b2c0-4f7f-8fd5-59a1c06ff2e4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.082526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2840-b919-4aaf-addf-172f71d0c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.083213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2840-b919-4aaf-addf-172f71d0c048 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.083474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2840-b919-4aaf-addf-172f71d0c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.083770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdf2840-b919-4aaf-addf-172f71d0c048 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.084467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf2840-b919-4aaf-addf-172f71d0c048 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.084682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61722e-f6b3-49f6-99ab-2007b4ce3aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.085899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61722e-f6b3-49f6-99ab-2007b4ce3aae admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.086292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61722e-f6b3-49f6-99ab-2007b4ce3aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.086566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdf2840-b919-4aaf-addf-172f71d0c048 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.087326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61722e-f6b3-49f6-99ab-2007b4ce3aae admin admin] PATH_INFO: `/v3/projects/ccf5c45783454fc5a7872490196f07e0/users/0cd8a309f6914d83b8e72bc60a5d7924/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.105768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61722e-f6b3-49f6-99ab-2007b4ce3aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf5c45783454fc5a7872490196f07e0, user_id=0cd8a309f6914d83b8e72bc60a5d7924, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.108456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c61722e-f6b3-49f6-99ab-2007b4ce3aae admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.110980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 789/1584] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:20 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:21.111359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:21.115409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2840-b919-4aaf-addf-172f71d0c048 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.126806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdf2840-b919-4aaf-addf-172f71d0c048 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.127569 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 796/1585] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:21 2022] PUT /identity/v3/projects/ccf5c45783454fc5a7872490196f07e0/users/0cd8a309f6914d83b8e72bc60a5d7924/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:21.128361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61722e-f6b3-49f6-99ab-2007b4ce3aae admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.202368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c132a-7684-4a63-89d5-7790f0c784b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.203058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c132a-7684-4a63-89d5-7790f0c784b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.203346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c132a-7684-4a63-89d5-7790f0c784b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.203621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c132a-7684-4a63-89d5-7790f0c784b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.204323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c132a-7684-4a63-89d5-7790f0c784b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.204676 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c61722e-f6b3-49f6-99ab-2007b4ce3aae admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.206297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c132a-7684-4a63-89d5-7790f0c784b4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.211878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475c57a-d9f8-4fc2-ae56-824641f0df00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.213056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c57a-d9f8-4fc2-ae56-824641f0df00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.213544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c57a-d9f8-4fc2-ae56-824641f0df00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.214393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c57a-d9f8-4fc2-ae56-824641f0df00 admin admin] PATH_INFO: `/v3/projects/667680b20ad541afa95e3f70b2f3cb16/users/1ea9428c59714013b28d8944506e6e2d/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.230537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 790/1586] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:21.230537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f2801-a928-4a0c-9906-4c5f5288d4ca None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.230537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f2801-a928-4a0c-9906-4c5f5288d4ca None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.230537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f2801-a928-4a0c-9906-4c5f5288d4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.232900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475c57a-d9f8-4fc2-ae56-824641f0df00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=667680b20ad541afa95e3f70b2f3cb16, user_id=1ea9428c59714013b28d8944506e6e2d, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.235600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475c57a-d9f8-4fc2-ae56-824641f0df00 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.250115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 797/1587] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:21 2022] PUT /identity/v3/projects/667680b20ad541afa95e3f70b2f3cb16/users/1ea9428c59714013b28d8944506e6e2d/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:21.252721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-408f2801-a928-4a0c-9906-4c5f5288d4ca None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:21.253974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-408f2801-a928-4a0c-9906-4c5f5288d4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:21.254318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475c57a-d9f8-4fc2-ae56-824641f0df00 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.315890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 791/1588] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:21.316621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f2801-a928-4a0c-9906-4c5f5288d4ca None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.326417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f2801-a928-4a0c-9906-4c5f5288d4ca None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.330538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475c57a-d9f8-4fc2-ae56-824641f0df00 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.331045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecb9f4-9ab9-4eef-8c80-e59a6df0989d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.331717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecb9f4-9ab9-4eef-8c80-e59a6df0989d admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.331992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecb9f4-9ab9-4eef-8c80-e59a6df0989d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.332267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecb9f4-9ab9-4eef-8c80-e59a6df0989d admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.332953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecb9f4-9ab9-4eef-8c80-e59a6df0989d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.334886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecb9f4-9ab9-4eef-8c80-e59a6df0989d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.338068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90298597-084c-4b41-9096-64f87943be34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.338937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90298597-084c-4b41-9096-64f87943be34 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.339306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90298597-084c-4b41-9096-64f87943be34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.339648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90298597-084c-4b41-9096-64f87943be34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.340546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90298597-084c-4b41-9096-64f87943be34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.342514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90298597-084c-4b41-9096-64f87943be34 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.355303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 798/1589] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.356262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90298597-084c-4b41-9096-64f87943be34 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.367778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 792/1590] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/users => generated 295 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:21.367778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecb9f4-9ab9-4eef-8c80-e59a6df0989d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.373398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90298597-084c-4b41-9096-64f87943be34 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.375488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecb9f4-9ab9-4eef-8c80-e59a6df0989d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.380216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c5d342-b177-4c7f-bda3-170ba636e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.380506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a6c0a-c974-4ccb-b4c1-7ec04873b9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.380893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5d342-b177-4c7f-bda3-170ba636e841 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.381193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5d342-b177-4c7f-bda3-170ba636e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.381491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c5d342-b177-4c7f-bda3-170ba636e841 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.381592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a6c0a-c974-4ccb-b4c1-7ec04873b9bf admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.382030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a6c0a-c974-4ccb-b4c1-7ec04873b9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.382229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5d342-b177-4c7f-bda3-170ba636e841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.382439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a6c0a-c974-4ccb-b4c1-7ec04873b9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.384253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a6c0a-c974-4ccb-b4c1-7ec04873b9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.384377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c5d342-b177-4c7f-bda3-170ba636e841 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.386921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a6c0a-c974-4ccb-b4c1-7ec04873b9bf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.397228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 793/1591] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.397228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494a26-fa5b-4186-be46-f50ba534b0f3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.397228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494a26-fa5b-4186-be46-f50ba534b0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.397228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f494a26-fa5b-4186-be46-f50ba534b0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.407247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 799/1592] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.407247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a6c0a-c974-4ccb-b4c1-7ec04873b9bf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.414885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a6c0a-c974-4ccb-b4c1-7ec04873b9bf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.416023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8f494a26-fa5b-4186-be46-f50ba534b0f3 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:21.417655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8f494a26-fa5b-4186-be46-f50ba534b0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:21.420445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4fa22-2d1a-4dd9-bfb2-4f2ad1c46486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.421115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4fa22-2d1a-4dd9-bfb2-4f2ad1c46486 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.421507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4fa22-2d1a-4dd9-bfb2-4f2ad1c46486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.421785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4fa22-2d1a-4dd9-bfb2-4f2ad1c46486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.422478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4fa22-2d1a-4dd9-bfb2-4f2ad1c46486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.424077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4fa22-2d1a-4dd9-bfb2-4f2ad1c46486 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.426541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 794/1593] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:21.427787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba26f5-6e8d-45b4-90e9-1788e06d0452 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.428070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba26f5-6e8d-45b4-90e9-1788e06d0452 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.428345 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba26f5-6e8d-45b4-90e9-1788e06d0452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.439888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 800/1594] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.440767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4fa22-2d1a-4dd9-bfb2-4f2ad1c46486 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.458730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cbba26f5-6e8d-45b4-90e9-1788e06d0452 None None] MFA Rules not processed for user `1ea9428c59714013b28d8944506e6e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:21.459936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cbba26f5-6e8d-45b4-90e9-1788e06d0452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:21.484903 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:21.516912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4fa22-2d1a-4dd9-bfb2-4f2ad1c46486 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.520862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b607e0-b42c-42c7-b790-64fef8b1086d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.521678 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b607e0-b42c-42c7-b790-64fef8b1086d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.521967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b607e0-b42c-42c7-b790-64fef8b1086d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.522247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b607e0-b42c-42c7-b790-64fef8b1086d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.530004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:21.537711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b607e0-b42c-42c7-b790-64fef8b1086d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.538770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 795/1595] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:21.539750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b607e0-b42c-42c7-b790-64fef8b1086d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.540562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba26f5-6e8d-45b4-90e9-1788e06d0452 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.550555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 801/1596] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:21.551388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b607e0-b42c-42c7-b790-64fef8b1086d service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.558118 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba26f5-6e8d-45b4-90e9-1788e06d0452 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.560672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b607e0-b42c-42c7-b790-64fef8b1086d service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.565150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c487-ead5-4696-89e1-0231726b3e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.565782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d36599-1ae2-4578-9985-6fcf933d4b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.566322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c487-ead5-4696-89e1-0231726b3e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.566650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36599-1ae2-4578-9985-6fcf933d4b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.566821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c487-ead5-4696-89e1-0231726b3e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.566970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36599-1ae2-4578-9985-6fcf933d4b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.567103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c487-ead5-4696-89e1-0231726b3e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.567241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d36599-1ae2-4578-9985-6fcf933d4b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.567947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d36599-1ae2-4578-9985-6fcf933d4b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.568277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41c487-ead5-4696-89e1-0231726b3e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.569580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d36599-1ae2-4578-9985-6fcf933d4b36 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.571773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41c487-ead5-4696-89e1-0231726b3e63 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.585278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 802/1597] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.585278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d36599-1ae2-4578-9985-6fcf933d4b36 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.591407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 796/1598] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.591407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c487-ead5-4696-89e1-0231726b3e63 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.594881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d36599-1ae2-4578-9985-6fcf933d4b36 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.600205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bc65e-32b3-4acb-985f-5923aa8ed755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.600900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bc65e-32b3-4acb-985f-5923aa8ed755 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.601214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bc65e-32b3-4acb-985f-5923aa8ed755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.601498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bc65e-32b3-4acb-985f-5923aa8ed755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.602305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990bc65e-32b3-4acb-985f-5923aa8ed755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.603966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990bc65e-32b3-4acb-985f-5923aa8ed755 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.609012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c487-ead5-4696-89e1-0231726b3e63 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.618386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 803/1599] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.618386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bc65e-32b3-4acb-985f-5923aa8ed755 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.620819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c7e93-4cea-438f-9ded-8ac4ce12740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.622263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c7e93-4cea-438f-9ded-8ac4ce12740b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.622637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c7e93-4cea-438f-9ded-8ac4ce12740b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.623391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c7e93-4cea-438f-9ded-8ac4ce12740b admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.623699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c7e93-4cea-438f-9ded-8ac4ce12740b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.625395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c7e93-4cea-438f-9ded-8ac4ce12740b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.634155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bc65e-32b3-4acb-985f-5923aa8ed755 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.637068 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 797/1600] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.637839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c7e93-4cea-438f-9ded-8ac4ce12740b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.642027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43bb86-8430-4390-99f9-0541e8378fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.643354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43bb86-8430-4390-99f9-0541e8378fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.644370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43bb86-8430-4390-99f9-0541e8378fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.645010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb43bb86-8430-4390-99f9-0541e8378fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.648055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c7e93-4cea-438f-9ded-8ac4ce12740b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.653708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873686b4-cd92-4d5b-8b17-fbcd6c9f157e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.654391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873686b4-cd92-4d5b-8b17-fbcd6c9f157e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.654680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873686b4-cd92-4d5b-8b17-fbcd6c9f157e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.654961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873686b4-cd92-4d5b-8b17-fbcd6c9f157e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.655747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873686b4-cd92-4d5b-8b17-fbcd6c9f157e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.657529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873686b4-cd92-4d5b-8b17-fbcd6c9f157e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.661867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43bb86-8430-4390-99f9-0541e8378fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.664666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb43bb86-8430-4390-99f9-0541e8378fb0 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.669382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 798/1601] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.670232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873686b4-cd92-4d5b-8b17-fbcd6c9f157e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.681632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 804/1602] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:21.682428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43bb86-8430-4390-99f9-0541e8378fb0 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.685444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873686b4-cd92-4d5b-8b17-fbcd6c9f157e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.690863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d5941-44a0-40db-ae9d-ee7fd45eab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.691275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d5941-44a0-40db-ae9d-ee7fd45eab52 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.691553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d5941-44a0-40db-ae9d-ee7fd45eab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.691831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d5941-44a0-40db-ae9d-ee7fd45eab52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.692563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d5941-44a0-40db-ae9d-ee7fd45eab52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.694584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d5941-44a0-40db-ae9d-ee7fd45eab52 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.695958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb43bb86-8430-4390-99f9-0541e8378fb0 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.701608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb591c-b3ec-4918-a8f6-9fb48e2f1321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.702473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb591c-b3ec-4918-a8f6-9fb48e2f1321 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.702801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb591c-b3ec-4918-a8f6-9fb48e2f1321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.703246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb591c-b3ec-4918-a8f6-9fb48e2f1321 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.704670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb591c-b3ec-4918-a8f6-9fb48e2f1321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.707973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb591c-b3ec-4918-a8f6-9fb48e2f1321 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.723163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 805/1603] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:21.724144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb591c-b3ec-4918-a8f6-9fb48e2f1321 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.730819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 799/1604] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:21 2022] POST /identity/v3/projects => generated 363 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:21.734604 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d5941-44a0-40db-ae9d-ee7fd45eab52 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.737649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb591c-b3ec-4918-a8f6-9fb48e2f1321 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.803713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbc01f-1a22-428a-8626-5743913500e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.804621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbc01f-1a22-428a-8626-5743913500e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.805293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbc01f-1a22-428a-8626-5743913500e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.805293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcbc01f-1a22-428a-8626-5743913500e0 admin admin] PATH_INFO: `/v3/projects/f029a8bb389c4d348c0e9001c5fa059c/users/1a3418bd5d7f430ca84efd5c83e38fed/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.823206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcbc01f-1a22-428a-8626-5743913500e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f029a8bb389c4d348c0e9001c5fa059c, user_id=1a3418bd5d7f430ca84efd5c83e38fed, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.826720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcbc01f-1a22-428a-8626-5743913500e0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.832237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d5941-44a0-40db-ae9d-ee7fd45eab52 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.845129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630f987-d87b-4362-af6c-7c15de860598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.849396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630f987-d87b-4362-af6c-7c15de860598 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.850285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630f987-d87b-4362-af6c-7c15de860598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.851594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 806/1605] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:21 2022] PUT /identity/v3/projects/f029a8bb389c4d348c0e9001c5fa059c/users/1a3418bd5d7f430ca84efd5c83e38fed/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:21.852067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630f987-d87b-4362-af6c-7c15de860598 admin admin] PATH_INFO: `/v3/projects/6a757117cfa64066b1931bf9e0b77836/users/a0a0cbb8e9284ac9a77e12e9f0f640a4/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:21.853199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbc01f-1a22-428a-8626-5743913500e0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.871778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630f987-d87b-4362-af6c-7c15de860598 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a757117cfa64066b1931bf9e0b77836, user_id=a0a0cbb8e9284ac9a77e12e9f0f640a4, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:21.874917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630f987-d87b-4362-af6c-7c15de860598 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:21.892998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 800/1606] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:21 2022] PUT /identity/v3/projects/6a757117cfa64066b1931bf9e0b77836/users/a0a0cbb8e9284ac9a77e12e9f0f640a4/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:21.894152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630f987-d87b-4362-af6c-7c15de860598 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:21.926066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcbc01f-1a22-428a-8626-5743913500e0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.969066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:21.983078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:21.984686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630f987-d87b-4362-af6c-7c15de860598 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:21.993938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b841208-1c8e-44d3-a963-ac76fb78b65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:21.995498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b841208-1c8e-44d3-a963-ac76fb78b65c admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:21.995780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b841208-1c8e-44d3-a963-ac76fb78b65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:21.996234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b841208-1c8e-44d3-a963-ac76fb78b65c admin admin] PATH_INFO: `/v3/projects/e2cbac6785114a3f87532884f214ab55/users/dac292a37ee14e35a63adf64201f1d2a/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.014609 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b841208-1c8e-44d3-a963-ac76fb78b65c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2cbac6785114a3f87532884f214ab55, user_id=dac292a37ee14e35a63adf64201f1d2a, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.018191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b841208-1c8e-44d3-a963-ac76fb78b65c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.026220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13f798-0739-46c2-a822-b76b7ed47596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:22.028598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f798-0739-46c2-a822-b76b7ed47596 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.029057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f798-0739-46c2-a822-b76b7ed47596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.029724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b13f798-0739-46c2-a822-b76b7ed47596 admin admin] PATH_INFO: `/v3/projects/87b7cb6d988f459c954da05d04a4f2c4/users/6cd6c5815f234ac1a651d4a2d4004563/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.037736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 801/1607] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:21 2022] PUT /identity/v3/projects/e2cbac6785114a3f87532884f214ab55/users/dac292a37ee14e35a63adf64201f1d2a/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:22.038641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b841208-1c8e-44d3-a963-ac76fb78b65c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:22.061782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13f798-0739-46c2-a822-b76b7ed47596 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87b7cb6d988f459c954da05d04a4f2c4, user_id=6cd6c5815f234ac1a651d4a2d4004563, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.065768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b13f798-0739-46c2-a822-b76b7ed47596 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.087013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 807/1608] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:21 2022] PUT /identity/v3/projects/87b7cb6d988f459c954da05d04a4f2c4/users/6cd6c5815f234ac1a651d4a2d4004563/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:22.089347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13f798-0739-46c2-a822-b76b7ed47596 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:22.158552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b841208-1c8e-44d3-a963-ac76fb78b65c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:22.218279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b13f798-0739-46c2-a822-b76b7ed47596 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:22.230767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb3806-5fa6-4ca9-9816-c9c1af8a2189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:22.231773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb3806-5fa6-4ca9-9816-c9c1af8a2189 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.232246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb3806-5fa6-4ca9-9816-c9c1af8a2189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.232623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb3806-5fa6-4ca9-9816-c9c1af8a2189 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.233461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb3806-5fa6-4ca9-9816-c9c1af8a2189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.236414 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb3806-5fa6-4ca9-9816-c9c1af8a2189 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.236984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e062595-8498-4963-8ba3-fe0a40aa3292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:22.238360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062595-8498-4963-8ba3-fe0a40aa3292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.238668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062595-8498-4963-8ba3-fe0a40aa3292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.238966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062595-8498-4963-8ba3-fe0a40aa3292 admin admin] PATH_INFO: `/v3/projects/0c63f648631a4a11910b4b135f22b235/users/68da2fd51cc84b2b8d641e1e6ac17c72/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.259281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062595-8498-4963-8ba3-fe0a40aa3292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c63f648631a4a11910b4b135f22b235, user_id=68da2fd51cc84b2b8d641e1e6ac17c72, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.263408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062595-8498-4963-8ba3-fe0a40aa3292 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.268967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 808/1609] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:22.268967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f13dd-4120-401a-8cfd-c4134ffa70e0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.268967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f13dd-4120-401a-8cfd-c4134ffa70e0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.268967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f13dd-4120-401a-8cfd-c4134ffa70e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.287701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 802/1610] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:22 2022] PUT /identity/v3/projects/0c63f648631a4a11910b4b135f22b235/users/68da2fd51cc84b2b8d641e1e6ac17c72/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:22.290358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d77d4-2ee1-4382-af0d-68b915bf2ddd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.290901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d77d4-2ee1-4382-af0d-68b915bf2ddd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.291425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d77d4-2ee1-4382-af0d-68b915bf2ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.295176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b73f13dd-4120-401a-8cfd-c4134ffa70e0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:22.296530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b73f13dd-4120-401a-8cfd-c4134ffa70e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:22.328612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-612d77d4-2ee1-4382-af0d-68b915bf2ddd None None] MFA Rules not processed for user `a0a0cbb8e9284ac9a77e12e9f0f640a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:22.330503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-612d77d4-2ee1-4382-af0d-68b915bf2ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:22.406276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:22.417356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 809/1611] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:22.420515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 803/1612] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:22.421993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd4e8-0442-406d-abc1-e324af035b1b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.422406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd4e8-0442-406d-abc1-e324af035b1b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.422757 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5fd4e8-0442-406d-abc1-e324af035b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.424623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a89f1-880e-4bff-9f77-cfdd6e2f1169 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.425062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a89f1-880e-4bff-9f77-cfdd6e2f1169 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.425357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a89f1-880e-4bff-9f77-cfdd6e2f1169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.495166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1a5fd4e8-0442-406d-abc1-e324af035b1b None None] MFA Rules not processed for user `6cd6c5815f234ac1a651d4a2d4004563`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:22.497046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1a5fd4e8-0442-406d-abc1-e324af035b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:22.500821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-796a89f1-880e-4bff-9f77-cfdd6e2f1169 None None] MFA Rules not processed for user `dac292a37ee14e35a63adf64201f1d2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:22.502060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-796a89f1-880e-4bff-9f77-cfdd6e2f1169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:22.562820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:22.608052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 804/1613] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:22.612698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fd4e8-0442-406d-abc1-e324af035b1b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:22.617116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 810/1614] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:22.619012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9177d-ed49-4cf2-9969-bc3314875b02 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.621006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9177d-ed49-4cf2-9969-bc3314875b02 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.621563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9177d-ed49-4cf2-9969-bc3314875b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.624848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5fd4e8-0442-406d-abc1-e324af035b1b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:22.630994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebba91e-10d9-40f5-a62d-ab349373faba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:22.633226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebba91e-10d9-40f5-a62d-ab349373faba admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.633226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebba91e-10d9-40f5-a62d-ab349373faba admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.633226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebba91e-10d9-40f5-a62d-ab349373faba admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.633581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebba91e-10d9-40f5-a62d-ab349373faba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.635167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebba91e-10d9-40f5-a62d-ab349373faba admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.646584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a5e9177d-ed49-4cf2-9969-bc3314875b02 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:22.648353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a5e9177d-ed49-4cf2-9969-bc3314875b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:22.652823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 805/1615] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:22.654038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebba91e-10d9-40f5-a62d-ab349373faba admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:22.658838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 811/1616] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:22.659656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9177d-ed49-4cf2-9969-bc3314875b02 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:22.661039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a5e9177d-ed49-4cf2-9969-bc3314875b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:22.668294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebba91e-10d9-40f5-a62d-ab349373faba admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:22.676772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5828d1-96bf-4902-a988-0bf5e7fc073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:22.680044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5828d1-96bf-4902-a988-0bf5e7fc073b admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.680344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5828d1-96bf-4902-a988-0bf5e7fc073b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.680632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5828d1-96bf-4902-a988-0bf5e7fc073b admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.681386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5828d1-96bf-4902-a988-0bf5e7fc073b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.683327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5828d1-96bf-4902-a988-0bf5e7fc073b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.735792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 806/1617] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:22.736664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5828d1-96bf-4902-a988-0bf5e7fc073b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:22.751606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9177d-ed49-4cf2-9969-bc3314875b02 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:22.830406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5828d1-96bf-4902-a988-0bf5e7fc073b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:22.842076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402bb90-4a4c-4034-9dab-ca6aeddcbd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:22.843481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402bb90-4a4c-4034-9dab-ca6aeddcbd03 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.844066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402bb90-4a4c-4034-9dab-ca6aeddcbd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.844596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402bb90-4a4c-4034-9dab-ca6aeddcbd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.845016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f921b33-35c2-4520-937b-5024734fb0a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:22.845780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402bb90-4a4c-4034-9dab-ca6aeddcbd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.845957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f921b33-35c2-4520-937b-5024734fb0a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:22.846324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f921b33-35c2-4520-937b-5024734fb0a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:22.846696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f921b33-35c2-4520-937b-5024734fb0a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:22.849259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402bb90-4a4c-4034-9dab-ca6aeddcbd03 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.873148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f921b33-35c2-4520-937b-5024734fb0a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:22.873880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f921b33-35c2-4520-937b-5024734fb0a4 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:22.886853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 812/1618] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:22.887713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f921b33-35c2-4520-937b-5024734fb0a4 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:22.901641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f921b33-35c2-4520-937b-5024734fb0a4 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:22.904338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 807/1619] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/projects => generated 405 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:22.908495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402bb90-4a4c-4034-9dab-ca6aeddcbd03 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:22.976338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:23.019017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402bb90-4a4c-4034-9dab-ca6aeddcbd03 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.029806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212310e3-ed12-47ad-b26a-c535ff2757ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.030714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212310e3-ed12-47ad-b26a-c535ff2757ef admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.031005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212310e3-ed12-47ad-b26a-c535ff2757ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.031388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212310e3-ed12-47ad-b26a-c535ff2757ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.032105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c27f2-c491-4c49-8188-a20a87caeefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.032243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212310e3-ed12-47ad-b26a-c535ff2757ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.033825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c27f2-c491-4c49-8188-a20a87caeefa admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.036112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212310e3-ed12-47ad-b26a-c535ff2757ef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.036255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c27f2-c491-4c49-8188-a20a87caeefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.036255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c27f2-c491-4c49-8188-a20a87caeefa admin admin] PATH_INFO: `/v3/projects/84e65fadd26141bfa86232b347104628/users/0ec4618fac9a4c84ba9ce008264c044e/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.073005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c27f2-c491-4c49-8188-a20a87caeefa admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e65fadd26141bfa86232b347104628, user_id=0ec4618fac9a4c84ba9ce008264c044e, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.076802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c27f2-c491-4c49-8188-a20a87caeefa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.077842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 813/1620] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:22 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:23.078491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212310e3-ed12-47ad-b26a-c535ff2757ef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.105838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 808/1621] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:22 2022] PUT /identity/v3/projects/84e65fadd26141bfa86232b347104628/users/0ec4618fac9a4c84ba9ce008264c044e/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:23.109339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:23.109760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c27f2-c491-4c49-8188-a20a87caeefa admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.170255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212310e3-ed12-47ad-b26a-c535ff2757ef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.257590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c27f2-c491-4c49-8188-a20a87caeefa admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.269517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0e76b-86e6-4538-965d-e75f681714eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.271062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0e76b-86e6-4538-965d-e75f681714eb admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.272165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0e76b-86e6-4538-965d-e75f681714eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.272165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa0e76b-86e6-4538-965d-e75f681714eb admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.274682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0e76b-86e6-4538-965d-e75f681714eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.279222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa0e76b-86e6-4538-965d-e75f681714eb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.287864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc1990-df91-49bc-a9b4-9e958fa76ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.290667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc1990-df91-49bc-a9b4-9e958fa76ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.291157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc1990-df91-49bc-a9b4-9e958fa76ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.291685 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc1990-df91-49bc-a9b4-9e958fa76ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.322319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc1990-df91-49bc-a9b4-9e958fa76ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.325831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc1990-df91-49bc-a9b4-9e958fa76ae1 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.330727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 809/1622] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:23 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:23.331744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0e76b-86e6-4538-965d-e75f681714eb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.371048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 814/1623] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:23.372370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc1990-df91-49bc-a9b4-9e958fa76ae1 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.376003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa0e76b-86e6-4538-965d-e75f681714eb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.387200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc1990-df91-49bc-a9b4-9e958fa76ae1 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.392198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5240af7-0651-4b12-90d3-40463b96ceed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.393774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5240af7-0651-4b12-90d3-40463b96ceed admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.394255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53465f-8ca2-4405-b93f-591c37e9643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.395035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53465f-8ca2-4405-b93f-591c37e9643b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.395288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5240af7-0651-4b12-90d3-40463b96ceed admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.395408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53465f-8ca2-4405-b93f-591c37e9643b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.395640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53465f-8ca2-4405-b93f-591c37e9643b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.395858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5240af7-0651-4b12-90d3-40463b96ceed admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.397410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53465f-8ca2-4405-b93f-591c37e9643b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.398385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5240af7-0651-4b12-90d3-40463b96ceed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.400875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5240af7-0651-4b12-90d3-40463b96ceed admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.401134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53465f-8ca2-4405-b93f-591c37e9643b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.505843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 815/1624] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:23 2022] POST /identity/v3/users => generated 300 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:23.505843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53465f-8ca2-4405-b93f-591c37e9643b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.510224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 810/1625] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:23 2022] POST /identity/v3/users => generated 309 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:23.510224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5240af7-0651-4b12-90d3-40463b96ceed admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.517164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53465f-8ca2-4405-b93f-591c37e9643b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.522709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7874018-409c-4e9f-9b3d-c0c673f003bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.523397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874018-409c-4e9f-9b3d-c0c673f003bc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.523785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874018-409c-4e9f-9b3d-c0c673f003bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.524693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874018-409c-4e9f-9b3d-c0c673f003bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.525509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7874018-409c-4e9f-9b3d-c0c673f003bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.528043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7874018-409c-4e9f-9b3d-c0c673f003bc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.535851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5240af7-0651-4b12-90d3-40463b96ceed admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.545605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 816/1626] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:23.549218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab618ed4-e828-47e0-8626-8fa218131e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.549586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7874018-409c-4e9f-9b3d-c0c673f003bc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.549982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618ed4-e828-47e0-8626-8fa218131e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.550209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618ed4-e828-47e0-8626-8fa218131e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.552742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab618ed4-e828-47e0-8626-8fa218131e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.553201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab618ed4-e828-47e0-8626-8fa218131e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.554914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab618ed4-e828-47e0-8626-8fa218131e17 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.561994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7874018-409c-4e9f-9b3d-c0c673f003bc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.572052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fb006-4bd8-4c37-b6d7-164146750eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.572869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fb006-4bd8-4c37-b6d7-164146750eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.573224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fb006-4bd8-4c37-b6d7-164146750eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.573558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7fb006-4bd8-4c37-b6d7-164146750eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.574819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fb006-4bd8-4c37-b6d7-164146750eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.576247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7fb006-4bd8-4c37-b6d7-164146750eb7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.587488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 811/1627] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:23.588419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab618ed4-e828-47e0-8626-8fa218131e17 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.596511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 817/1628] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:23.596511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fb006-4bd8-4c37-b6d7-164146750eb7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.611794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7fb006-4bd8-4c37-b6d7-164146750eb7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.615794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab618ed4-e828-47e0-8626-8fa218131e17 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.616699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafd777-e4d8-4a10-8b84-13a03e6a5669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.618086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafd777-e4d8-4a10-8b84-13a03e6a5669 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.618253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafd777-e4d8-4a10-8b84-13a03e6a5669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.618609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bafd777-e4d8-4a10-8b84-13a03e6a5669 admin admin] PATH_INFO: `/v3/projects/84e65fadd26141bfa86232b347104628/users/0ec4618fac9a4c84ba9ce008264c044e/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.621697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafd777-e4d8-4a10-8b84-13a03e6a5669 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e65fadd26141bfa86232b347104628, user_id=0ec4618fac9a4c84ba9ce008264c044e, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.625016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bafd777-e4d8-4a10-8b84-13a03e6a5669 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.629360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058547bb-0bce-47db-abf8-14c904d181c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.630579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058547bb-0bce-47db-abf8-14c904d181c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.630579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058547bb-0bce-47db-abf8-14c904d181c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.630780 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058547bb-0bce-47db-abf8-14c904d181c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.631506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058547bb-0bce-47db-abf8-14c904d181c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.633646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058547bb-0bce-47db-abf8-14c904d181c4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.648669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 812/1629] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:23.648669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058547bb-0bce-47db-abf8-14c904d181c4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.656315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 818/1630] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:23 2022] PUT /identity/v3/projects/84e65fadd26141bfa86232b347104628/users/0ec4618fac9a4c84ba9ce008264c044e/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:23.657364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafd777-e4d8-4a10-8b84-13a03e6a5669 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.658028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058547bb-0bce-47db-abf8-14c904d181c4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.730127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b0b01-6d08-4d7b-b3db-80aa921bdcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.731099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0b01-6d08-4d7b-b3db-80aa921bdcc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.732077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0b01-6d08-4d7b-b3db-80aa921bdcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.732374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829b0b01-6d08-4d7b-b3db-80aa921bdcc4 admin admin] PATH_INFO: `/v3/projects/8c905938a7c243e59de2059e8f5f62e2/users/e987aad5c9d84af68dfc5943ea5fe5aa/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.752561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b0b01-6d08-4d7b-b3db-80aa921bdcc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c905938a7c243e59de2059e8f5f62e2, user_id=e987aad5c9d84af68dfc5943ea5fe5aa, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.756082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829b0b01-6d08-4d7b-b3db-80aa921bdcc4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.759504 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:23.781061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 813/1631] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:23 2022] PUT /identity/v3/projects/8c905938a7c243e59de2059e8f5f62e2/users/e987aad5c9d84af68dfc5943ea5fe5aa/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:23.782195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b0b01-6d08-4d7b-b3db-80aa921bdcc4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:23.802038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bafd777-e4d8-4a10-8b84-13a03e6a5669 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.870523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829b0b01-6d08-4d7b-b3db-80aa921bdcc4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:23.876678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ddb26c-ce46-4a69-b2b1-0f3adf35e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.877641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb26c-ce46-4a69-b2b1-0f3adf35e056 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.877930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb26c-ce46-4a69-b2b1-0f3adf35e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.878269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ddb26c-ce46-4a69-b2b1-0f3adf35e056 admin admin] PATH_INFO: `/v3/projects/904fabbf81b748d98bbc0ff8b30f3f67/users/896781a191754fa58e5d3d2e440435bf/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.884321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:23.901222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddb26c-ce46-4a69-b2b1-0f3adf35e056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=904fabbf81b748d98bbc0ff8b30f3f67, user_id=896781a191754fa58e5d3d2e440435bf, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.904707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ddb26c-ce46-4a69-b2b1-0f3adf35e056 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:23.930372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 814/1632] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:23 2022] PUT /identity/v3/projects/904fabbf81b748d98bbc0ff8b30f3f67/users/896781a191754fa58e5d3d2e440435bf/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:23.931875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9202ed-ea02-4b19-9497-41554a38409e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.933809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9202ed-ea02-4b19-9497-41554a38409e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.934103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9202ed-ea02-4b19-9497-41554a38409e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.949865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e836db9-139b-4350-b3e3-87618e9f5bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:23.951565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e836db9-139b-4350-b3e3-87618e9f5bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:23.951861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e836db9-139b-4350-b3e3-87618e9f5bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:23.952137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e836db9-139b-4350-b3e3-87618e9f5bbb admin admin] PATH_INFO: `/v3/projects/b1f7c3b6d1844745970d39228ba80598/users/6051219233d24da4a1b8f42dda357204/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:23.991509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e836db9-139b-4350-b3e3-87618e9f5bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f7c3b6d1844745970d39228ba80598, user_id=6051219233d24da4a1b8f42dda357204, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:23.994638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e836db9-139b-4350-b3e3-87618e9f5bbb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.000282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8a9202ed-ea02-4b19-9497-41554a38409e None None] MFA Rules not processed for user `0ec4618fac9a4c84ba9ce008264c044e`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:24.001694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8a9202ed-ea02-4b19-9497-41554a38409e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:24.025520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 819/1633] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:23 2022] PUT /identity/v3/projects/b1f7c3b6d1844745970d39228ba80598/users/6051219233d24da4a1b8f42dda357204/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:24.025520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e836db9-139b-4350-b3e3-87618e9f5bbb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.101856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 815/1634] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:23 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:24.103753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9202ed-ea02-4b19-9497-41554a38409e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.185631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9202ed-ea02-4b19-9497-41554a38409e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.187398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e836db9-139b-4350-b3e3-87618e9f5bbb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.192177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba0128-44e6-48ea-8ee2-25468aec0421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.194117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba0128-44e6-48ea-8ee2-25468aec0421 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.194117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba0128-44e6-48ea-8ee2-25468aec0421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.194333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba0128-44e6-48ea-8ee2-25468aec0421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.194998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba0128-44e6-48ea-8ee2-25468aec0421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.197135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba0128-44e6-48ea-8ee2-25468aec0421 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.199540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.203221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a75c2f-70c5-458e-882a-42e1664010c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.203812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a75c2f-70c5-458e-882a-42e1664010c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.204107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a75c2f-70c5-458e-882a-42e1664010c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.204390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a75c2f-70c5-458e-882a-42e1664010c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.205141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a75c2f-70c5-458e-882a-42e1664010c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.207406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a75c2f-70c5-458e-882a-42e1664010c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.217032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 816/1635] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:24.220041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba0128-44e6-48ea-8ee2-25468aec0421 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.231510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 820/1636] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:24.232224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a75c2f-70c5-458e-882a-42e1664010c9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.240078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba0128-44e6-48ea-8ee2-25468aec0421 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.249239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9441777-c8f1-4e03-acb1-a0f261a57399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.251735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9441777-c8f1-4e03-acb1-a0f261a57399 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.252275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9441777-c8f1-4e03-acb1-a0f261a57399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.252873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9441777-c8f1-4e03-acb1-a0f261a57399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.253883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9441777-c8f1-4e03-acb1-a0f261a57399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.257354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9441777-c8f1-4e03-acb1-a0f261a57399 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.277883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 817/1637] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:24.279046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9441777-c8f1-4e03-acb1-a0f261a57399 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.327785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9441777-c8f1-4e03-acb1-a0f261a57399 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.337768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec543d-a16f-4bdd-b9c4-6b144cc5cf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.340516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec543d-a16f-4bdd-b9c4-6b144cc5cf51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.341404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec543d-a16f-4bdd-b9c4-6b144cc5cf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.342161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec543d-a16f-4bdd-b9c4-6b144cc5cf51 admin admin] PATH_INFO: `/v3/projects/904fabbf81b748d98bbc0ff8b30f3f67/users/896781a191754fa58e5d3d2e440435bf/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.346535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec543d-a16f-4bdd-b9c4-6b144cc5cf51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=904fabbf81b748d98bbc0ff8b30f3f67, user_id=896781a191754fa58e5d3d2e440435bf, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.366255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec543d-a16f-4bdd-b9c4-6b144cc5cf51 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.395149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a75c2f-70c5-458e-882a-42e1664010c9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.406288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 818/1638] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:24 2022] PUT /identity/v3/projects/904fabbf81b748d98bbc0ff8b30f3f67/users/896781a191754fa58e5d3d2e440435bf/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:24.411541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec543d-a16f-4bdd-b9c4-6b144cc5cf51 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.517685 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44010373-86a6-4888-abb8-2e690754dc2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.518660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010373-86a6-4888-abb8-2e690754dc2b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.519090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010373-86a6-4888-abb8-2e690754dc2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.519532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44010373-86a6-4888-abb8-2e690754dc2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.538237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec543d-a16f-4bdd-b9c4-6b144cc5cf51 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.545493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cf3cd-cc10-47eb-ab8d-5220d70a9dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.546653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cf3cd-cc10-47eb-ab8d-5220d70a9dd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.546978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cf3cd-cc10-47eb-ab8d-5220d70a9dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.547497 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883cf3cd-cc10-47eb-ab8d-5220d70a9dd8 admin admin] PATH_INFO: `/v3/projects/8c905938a7c243e59de2059e8f5f62e2/users/e987aad5c9d84af68dfc5943ea5fe5aa/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.549599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cf3cd-cc10-47eb-ab8d-5220d70a9dd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c905938a7c243e59de2059e8f5f62e2, user_id=e987aad5c9d84af68dfc5943ea5fe5aa, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.552771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883cf3cd-cc10-47eb-ab8d-5220d70a9dd8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.560239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:24.585864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 819/1639] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:24 2022] PUT /identity/v3/projects/8c905938a7c243e59de2059e8f5f62e2/users/e987aad5c9d84af68dfc5943ea5fe5aa/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:24.585864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cf3cd-cc10-47eb-ab8d-5220d70a9dd8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:24.674393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44010373-86a6-4888-abb8-2e690754dc2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.678434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44010373-86a6-4888-abb8-2e690754dc2b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.713571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 821/1640] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:24.716822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb39e2-b2f4-4838-bc72-bd5e80b987f9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.717357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb39e2-b2f4-4838-bc72-bd5e80b987f9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.718394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acb39e2-b2f4-4838-bc72-bd5e80b987f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.724091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883cf3cd-cc10-47eb-ab8d-5220d70a9dd8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:24.747677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5a411-bc6c-4986-b6c3-0974b27bd210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:24.749566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a411-bc6c-4986-b6c3-0974b27bd210 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.750182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a411-bc6c-4986-b6c3-0974b27bd210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.750638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5a411-bc6c-4986-b6c3-0974b27bd210 admin admin] PATH_INFO: `/v3/projects/b1f7c3b6d1844745970d39228ba80598/users/6051219233d24da4a1b8f42dda357204/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.762244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5a411-bc6c-4986-b6c3-0974b27bd210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f7c3b6d1844745970d39228ba80598, user_id=6051219233d24da4a1b8f42dda357204, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:24.766542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5a411-bc6c-4986-b6c3-0974b27bd210 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:24.794397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5acb39e2-b2f4-4838-bc72-bd5e80b987f9 None None] MFA Rules not processed for user `896781a191754fa58e5d3d2e440435bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:24.797779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5acb39e2-b2f4-4838-bc72-bd5e80b987f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:24.801124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 820/1641] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:24 2022] PUT /identity/v3/projects/b1f7c3b6d1844745970d39228ba80598/users/6051219233d24da4a1b8f42dda357204/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:24.803824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204bebb-b44c-4991-b507-03587a1e7b9e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:24.804131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204bebb-b44c-4991-b507-03587a1e7b9e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:24.804400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204bebb-b44c-4991-b507-03587a1e7b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:24.925990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c204bebb-b44c-4991-b507-03587a1e7b9e None None] MFA Rules not processed for user `e987aad5c9d84af68dfc5943ea5fe5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:24.925990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c204bebb-b44c-4991-b507-03587a1e7b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:25.047178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 822/1642] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:25.048585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceaf05-e549-45fe-9763-2bda4cb1d947 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.048873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceaf05-e549-45fe-9763-2bda4cb1d947 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.049156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ceaf05-e549-45fe-9763-2bda4cb1d947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.141870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:25.168347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b5ceaf05-e549-45fe-9763-2bda4cb1d947 None None] MFA Rules not processed for user `6051219233d24da4a1b8f42dda357204`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:25.172356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b5ceaf05-e549-45fe-9763-2bda4cb1d947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:25.213341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 821/1643] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:24 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:25.214367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204bebb-b44c-4991-b507-03587a1e7b9e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.366917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204bebb-b44c-4991-b507-03587a1e7b9e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.371879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30e79a-fac3-4479-830e-9c6ed56fcabd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.372567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e79a-fac3-4479-830e-9c6ed56fcabd service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.372855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e79a-fac3-4479-830e-9c6ed56fcabd service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.373138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30e79a-fac3-4479-830e-9c6ed56fcabd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.382137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 823/1644] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:25.383615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceaf05-e549-45fe-9763-2bda4cb1d947 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.398516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e30e79a-fac3-4479-830e-9c6ed56fcabd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.403201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e30e79a-fac3-4479-830e-9c6ed56fcabd service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.406200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ceaf05-e549-45fe-9763-2bda4cb1d947 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.411570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c0290-0993-4b0d-ba09-41ace0831bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.412504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c0290-0993-4b0d-ba09-41ace0831bed service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.412989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c0290-0993-4b0d-ba09-41ace0831bed service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.413528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c0290-0993-4b0d-ba09-41ace0831bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.434082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 822/1645] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:25 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:25.436503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c0290-0993-4b0d-ba09-41ace0831bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.441303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c0290-0993-4b0d-ba09-41ace0831bed service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.463708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 824/1646] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:25 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:25.597917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30e79a-fac3-4479-830e-9c6ed56fcabd service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.623593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30e79a-fac3-4479-830e-9c6ed56fcabd service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:25.632023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11988564-1926-4c0d-87f4-9fb9eded7dbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:25.633462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988564-1926-4c0d-87f4-9fb9eded7dbc service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:25.634784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988564-1926-4c0d-87f4-9fb9eded7dbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:25.635318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11988564-1926-4c0d-87f4-9fb9eded7dbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:25.660325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11988564-1926-4c0d-87f4-9fb9eded7dbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:25.664794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11988564-1926-4c0d-87f4-9fb9eded7dbc service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:25.695335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 823/1647] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:25.766720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c0290-0993-4b0d-ba09-41ace0831bed service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:25.810728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:25.983779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c0290-0993-4b0d-ba09-41ace0831bed service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:26.003134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:26.011153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.012153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.016754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] PATH_INFO: `/v3/users/dac292a37ee14e35a63adf64201f1d2a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.021559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac292a37ee14e35a63adf64201f1d2a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:26.027841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:26.033467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11988564-1926-4c0d-87f4-9fb9eded7dbc service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:26.056219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11988564-1926-4c0d-87f4-9fb9eded7dbc service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:26.080930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:26.086963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.091702 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.092477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] PATH_INFO: `/v3/users/6cd6c5815f234ac1a651d4a2d4004563` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.100077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cd6c5815f234ac1a651d4a2d4004563)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:26.107651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: WARNING py.warnings [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:22:26.107651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: warnings.warn(msg) Jun 16 17:22:26.107651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: Jun 16 17:22:26.119952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:26.263829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac292a37ee14e35a63adf64201f1d2a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.266269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac292a37ee14e35a63adf64201f1d2a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.268711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac292a37ee14e35a63adf64201f1d2a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.310395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac292a37ee14e35a63adf64201f1d2a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.327357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-633f0d70-4d50-42b4-bf4f-52c37347ec46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac292a37ee14e35a63adf64201f1d2a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.362843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cd6c5815f234ac1a651d4a2d4004563'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.369776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 825/1648] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:25 2022] DELETE /identity/v3/users/dac292a37ee14e35a63adf64201f1d2a => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:26.388994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbea0f3-139c-4d7b-b949-fdeefed9943f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.395604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbea0f3-139c-4d7b-b949-fdeefed9943f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.397885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbea0f3-139c-4d7b-b949-fdeefed9943f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.456727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cd6c5815f234ac1a651d4a2d4004563'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.459980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cd6c5815f234ac1a651d4a2d4004563'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.511421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5dbea0f3-139c-4d7b-b949-fdeefed9943f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:26.512887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5dbea0f3-139c-4d7b-b949-fdeefed9943f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:26.529744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cd6c5815f234ac1a651d4a2d4004563'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.568111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cd6c5815f234ac1a651d4a2d4004563'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:26.590335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 824/1649] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:26 2022] DELETE /identity/v3/users/6cd6c5815f234ac1a651d4a2d4004563 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:22:26.591632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:26.607448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:26.875813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 826/1650] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:26.880548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b2212-0f1b-450a-bbeb-86b45060d6be None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.881548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b2212-0f1b-450a-bbeb-86b45060d6be None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.882126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b2212-0f1b-450a-bbeb-86b45060d6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.888879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:26.936630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6de3f0-428e-41cd-b5a0-22ea89182def admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:26.956846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ff4b2212-0f1b-450a-bbeb-86b45060d6be None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:26.958494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ff4b2212-0f1b-450a-bbeb-86b45060d6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:26.964913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:26.966770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:26.968442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:26.969442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] PATH_INFO: `/v3/users/1ea9428c59714013b28d8944506e6e2d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:26.972610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea9428c59714013b28d8944506e6e2d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:26.975999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:26.984571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 827/1651] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:26.987415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b2212-0f1b-450a-bbeb-86b45060d6be None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:26.989096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ff4b2212-0f1b-450a-bbeb-86b45060d6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.019076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b2212-0f1b-450a-bbeb-86b45060d6be None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.036421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.037770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.038842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.039520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] PATH_INFO: `/v3/users/a0a0cbb8e9284ac9a77e12e9f0f640a4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.041538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0a0cbb8e9284ac9a77e12e9f0f640a4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.044073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.084655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea9428c59714013b28d8944506e6e2d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.108575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:27.116363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea9428c59714013b28d8944506e6e2d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.118339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea9428c59714013b28d8944506e6e2d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.142311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea9428c59714013b28d8944506e6e2d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.156635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0a0cbb8e9284ac9a77e12e9f0f640a4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.157795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0a0cbb8e9284ac9a77e12e9f0f640a4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.158855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0a0cbb8e9284ac9a77e12e9f0f640a4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.162446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea9428c59714013b28d8944506e6e2d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.175350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 825/1652] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:26 2022] DELETE /identity/v3/users/1ea9428c59714013b28d8944506e6e2d => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:27.177868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.182235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.198171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0a0cbb8e9284ac9a77e12e9f0f640a4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.229930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0a0cbb8e9284ac9a77e12e9f0f640a4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:27.278323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 828/1653] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:26 2022] DELETE /identity/v3/users/a0a0cbb8e9284ac9a77e12e9f0f640a4 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:27.279825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.285959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.371241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b1528-3873-4abb-bfab-9df5f9fd06eb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.441962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0b20-3f3c-41dd-a84f-10c13b6d8b3e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.457529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5d6fc-cdc8-4eb8-8db3-259b285c744c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.458685 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d6fc-cdc8-4eb8-8db3-259b285c744c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.459075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d6fc-cdc8-4eb8-8db3-259b285c744c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.461206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5d6fc-cdc8-4eb8-8db3-259b285c744c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.465385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6cc5d6fc-cdc8-4eb8-8db3-259b285c744c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.525232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.525856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.526148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.526516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.534847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.631630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:27.641856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5d6fc-cdc8-4eb8-8db3-259b285c744c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.644458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc5d6fc-cdc8-4eb8-8db3-259b285c744c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.667080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 829/1654] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:27.670005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaecb1-0170-4579-bdde-2218bc882cb9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.670606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaecb1-0170-4579-bdde-2218bc882cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.671095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eaecb1-0170-4579-bdde-2218bc882cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.704342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-08eaecb1-0170-4579-bdde-2218bc882cb9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:27.706377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.707282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-08eaecb1-0170-4579-bdde-2218bc882cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.707856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.719911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 830/1655] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:27.722570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b040454-b1ea-4b9f-b40b-e3dbb0e41e42 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.723306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b040454-b1ea-4b9f-b40b-e3dbb0e41e42 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.725407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b040454-b1ea-4b9f-b40b-e3dbb0e41e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.749580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 826/1656] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:27.751910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9b040454-b1ea-4b9f-b40b-e3dbb0e41e42 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:27.753722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9b040454-b1ea-4b9f-b40b-e3dbb0e41e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.770571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 831/1657] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:27.782668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.784707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:27.812647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8ff5b-5be7-4f95-a022-a3d26da0bd95 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.817014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.817713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.818034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.818274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.840371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.842112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.862599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b040454-b1ea-4b9f-b40b-e3dbb0e41e42 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:27.864760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 827/1658] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:27.879424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b040454-b1ea-4b9f-b40b-e3dbb0e41e42 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:27.890379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784291b9-663d-46f1-a836-06cf0ed9683c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:27.893016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784291b9-663d-46f1-a836-06cf0ed9683c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:27.895051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784291b9-663d-46f1-a836-06cf0ed9683c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:27.895933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784291b9-663d-46f1-a836-06cf0ed9683c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:27.929793 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784291b9-663d-46f1-a836-06cf0ed9683c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:27.934743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784291b9-663d-46f1-a836-06cf0ed9683c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:27.969643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 832/1659] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:29.148413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.150784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:29.181733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03797da-ee88-40e4-967f-7d85f7a91b13 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.200252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:29.203128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:29.203676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:29.204534 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] PATH_INFO: `/v3/projects/e2cbac6785114a3f87532884f214ab55` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:29.210896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2cbac6785114a3f87532884f214ab55)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:29.215242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:29.335748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2cbac6785114a3f87532884f214ab55', 'request_id': 'req-d100ab03-d36f-4afd-aa0b-701bb00c3698'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.349748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2cbac6785114a3f87532884f214ab55', 'request_id': 'req-d100ab03-d36f-4afd-aa0b-701bb00c3698'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.368945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2cbac6785114a3f87532884f214ab55', 'request_id': 'req-d100ab03-d36f-4afd-aa0b-701bb00c3698'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.397836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] The token cache is being invalidate because project e2cbac6785114a3f87532884f214ab55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:29.398144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.400333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.405431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 828/1660] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:29 2022] DELETE /identity/v3/projects/e2cbac6785114a3f87532884f214ab55 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:29.409718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784291b9-663d-46f1-a836-06cf0ed9683c service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.435395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784291b9-663d-46f1-a836-06cf0ed9683c service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.485620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:29.556518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:29.557550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:29.557907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:29.558296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] PATH_INFO: `/v3/users/6051219233d24da4a1b8f42dda357204` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:29.559796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6051219233d24da4a1b8f42dda357204)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:29.562373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:29.648080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6051219233d24da4a1b8f42dda357204'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.648944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6051219233d24da4a1b8f42dda357204'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.649993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6051219233d24da4a1b8f42dda357204'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.686626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6051219233d24da4a1b8f42dda357204'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.706666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6051219233d24da4a1b8f42dda357204'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:29.742742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 833/1661] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:29 2022] DELETE /identity/v3/users/6051219233d24da4a1b8f42dda357204 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:22:29.803795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.806530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:29.833426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100ab03-d36f-4afd-aa0b-701bb00c3698 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:29.971972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:29.974324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:30.004815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.007244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.011100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.019670 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] PATH_INFO: `/v3/projects/87b7cb6d988f459c954da05d04a4f2c4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.024656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87b7cb6d988f459c954da05d04a4f2c4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.030921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.158247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.163178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87b7cb6d988f459c954da05d04a4f2c4', 'request_id': 'req-ca296085-f59b-470d-89ca-cdca859d6711'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.186512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87b7cb6d988f459c954da05d04a4f2c4', 'request_id': 'req-ca296085-f59b-470d-89ca-cdca859d6711'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.196385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87b7cb6d988f459c954da05d04a4f2c4', 'request_id': 'req-ca296085-f59b-470d-89ca-cdca859d6711'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.216722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] The token cache is being invalidate because project 87b7cb6d988f459c954da05d04a4f2c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:30.217389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.218355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.220653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 829/1662] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:29 2022] DELETE /identity/v3/projects/87b7cb6d988f459c954da05d04a4f2c4 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:30.221939 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968fdf93-4fac-4b26-a5ba-62d3b1c9905b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.224545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.228214 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:30.310351 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.339223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.340777 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.341239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.341758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] PATH_INFO: `/v3/projects/667680b20ad541afa95e3f70b2f3cb16` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.343297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] RBAC: Authorizing `identity:delete_project(project_id=667680b20ad541afa95e3f70b2f3cb16)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.344736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: WARNING py.warnings [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 17:22:30.344736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: warnings.warn(msg) Jun 16 17:22:30.344736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: Jun 16 17:22:30.346330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.347018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca296085-f59b-470d-89ca-cdca859d6711 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.352522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.354040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.354432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.355275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] PATH_INFO: `/v3/users/896781a191754fa58e5d3d2e440435bf` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.356829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=896781a191754fa58e5d3d2e440435bf)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.358939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.455835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '896781a191754fa58e5d3d2e440435bf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.480950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '896781a191754fa58e5d3d2e440435bf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.482721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '896781a191754fa58e5d3d2e440435bf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.490594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '667680b20ad541afa95e3f70b2f3cb16', 'request_id': 'req-6399cac5-94ba-4515-ad94-a683dfbf6743'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.497280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '896781a191754fa58e5d3d2e440435bf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.501169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '667680b20ad541afa95e3f70b2f3cb16', 'request_id': 'req-6399cac5-94ba-4515-ad94-a683dfbf6743'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.509899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '896781a191754fa58e5d3d2e440435bf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.523235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 830/1663] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:30 2022] DELETE /identity/v3/users/896781a191754fa58e5d3d2e440435bf => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:30.531467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '667680b20ad541afa95e3f70b2f3cb16', 'request_id': 'req-6399cac5-94ba-4515-ad94-a683dfbf6743'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.562160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] The token cache is being invalidate because project 667680b20ad541afa95e3f70b2f3cb16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:30.562610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.563694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.568424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 834/1664] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:29 2022] DELETE /identity/v3/projects/667680b20ad541afa95e3f70b2f3cb16 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:22:30.569993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.572001 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:30.647853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:30.650383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:30.686423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:30.730161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d51a02-02d3-4c1b-99ff-14ba437186b7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.741359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.742567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.743608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.744001 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] PATH_INFO: `/v3/users/0ec4618fac9a4c84ba9ce008264c044e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.746515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec4618fac9a4c84ba9ce008264c044e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.754486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.849914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399cac5-94ba-4515-ad94-a683dfbf6743 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:30.858631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec4618fac9a4c84ba9ce008264c044e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.863977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:30.865304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:30.865608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:30.865887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] PATH_INFO: `/v3/projects/6a757117cfa64066b1931bf9e0b77836` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:30.870303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a757117cfa64066b1931bf9e0b77836)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:30.872297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:30.891955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec4618fac9a4c84ba9ce008264c044e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.895462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec4618fac9a4c84ba9ce008264c044e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.922972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec4618fac9a4c84ba9ce008264c044e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.942761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec4618fac9a4c84ba9ce008264c044e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:30.960234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 831/1665] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:30 2022] DELETE /identity/v3/users/0ec4618fac9a4c84ba9ce008264c044e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:31.002885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a757117cfa64066b1931bf9e0b77836', 'request_id': 'req-fd7620f8-5b2b-4554-9486-af61e88b9316'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.011314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a757117cfa64066b1931bf9e0b77836', 'request_id': 'req-fd7620f8-5b2b-4554-9486-af61e88b9316'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.041799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a757117cfa64066b1931bf9e0b77836', 'request_id': 'req-fd7620f8-5b2b-4554-9486-af61e88b9316'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.075223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] The token cache is being invalidate because project 6a757117cfa64066b1931bf9e0b77836 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:31.075514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.076386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.080673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 835/1666] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:30 2022] DELETE /identity/v3/projects/6a757117cfa64066b1931bf9e0b77836 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:31.088747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:31.092481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:31.274744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe97a0-64c8-4b09-857b-00e30c8f8049 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:31.401527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:31.402611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:31.404592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:31.405096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] PATH_INFO: `/v3/users/e987aad5c9d84af68dfc5943ea5fe5aa` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:31.408740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e987aad5c9d84af68dfc5943ea5fe5aa)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:31.411103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:31.457935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:31.545295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e987aad5c9d84af68dfc5943ea5fe5aa'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.579436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e987aad5c9d84af68dfc5943ea5fe5aa'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.580778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e987aad5c9d84af68dfc5943ea5fe5aa'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.593132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e987aad5c9d84af68dfc5943ea5fe5aa'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.612963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cf931d3c-29b6-4462-89e9-ba3c8388cdb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e987aad5c9d84af68dfc5943ea5fe5aa'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:31.623827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 832/1667] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:31 2022] DELETE /identity/v3/users/e987aad5c9d84af68dfc5943ea5fe5aa => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:32.205023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:32.214990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:32.255632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7620f8-5b2b-4554-9486-af61e88b9316 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:32.280367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:32.506426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:32.507477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:32.507860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:32.512490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] PATH_INFO: `/v3/projects/b1f7c3b6d1844745970d39228ba80598` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:32.514760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f7c3b6d1844745970d39228ba80598)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:32.517877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:32.686768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f7c3b6d1844745970d39228ba80598', 'request_id': 'req-23d53c91-9b1c-4673-80c2-5a993aaf316c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:32.697522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f7c3b6d1844745970d39228ba80598', 'request_id': 'req-23d53c91-9b1c-4673-80c2-5a993aaf316c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:32.716127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f7c3b6d1844745970d39228ba80598', 'request_id': 'req-23d53c91-9b1c-4673-80c2-5a993aaf316c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:32.753036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] The token cache is being invalidate because project b1f7c3b6d1844745970d39228ba80598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:32.753404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:32.754910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:32.759880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 836/1668] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:32 2022] DELETE /identity/v3/projects/b1f7c3b6d1844745970d39228ba80598 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:32.764736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685896e2-d674-4425-89a1-e666b21cd453 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:32.765435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685896e2-d674-4425-89a1-e666b21cd453 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:32.765435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685896e2-d674-4425-89a1-e666b21cd453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:32.820979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-685896e2-d674-4425-89a1-e666b21cd453 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:32.826717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-685896e2-d674-4425-89a1-e666b21cd453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:32.934694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:32.936290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.024408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 833/1669] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:33.026191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb8842-9915-432e-96b7-2360c50b4156 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.028109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb8842-9915-432e-96b7-2360c50b4156 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.028425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb8842-9915-432e-96b7-2360c50b4156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.069526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5ebb8842-9915-432e-96b7-2360c50b4156 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:33.075645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5ebb8842-9915-432e-96b7-2360c50b4156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.108196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 834/1670] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:33.111419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d53c91-9b1c-4673-80c2-5a993aaf316c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.112783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb8842-9915-432e-96b7-2360c50b4156 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.125062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb8842-9915-432e-96b7-2360c50b4156 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.131197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.131886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.132166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.132316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.132450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.133478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:33.133743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.134129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.134244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.134600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] PATH_INFO: `/v3/projects/8c905938a7c243e59de2059e8f5f62e2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.136624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c905938a7c243e59de2059e8f5f62e2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.137636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.140015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.157546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 835/1671] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:33.162427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.184574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1affe-f240-4c69-a113-690d478a4d85 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.190043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c3e4f-4ddc-44a3-b753-83130dd49f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.190868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c3e4f-4ddc-44a3-b753-83130dd49f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.190982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c3e4f-4ddc-44a3-b753-83130dd49f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.191237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c3e4f-4ddc-44a3-b753-83130dd49f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.192406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c3e4f-4ddc-44a3-b753-83130dd49f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.194676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c3e4f-4ddc-44a3-b753-83130dd49f22 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.253843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.258448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 836/1672] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:33 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:33.262841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c3e4f-4ddc-44a3-b753-83130dd49f22 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.263482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c905938a7c243e59de2059e8f5f62e2', 'request_id': 'req-82027d7e-f2e0-4f6f-b790-7b82cba47631'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.276403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c905938a7c243e59de2059e8f5f62e2', 'request_id': 'req-82027d7e-f2e0-4f6f-b790-7b82cba47631'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.297290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c905938a7c243e59de2059e8f5f62e2', 'request_id': 'req-82027d7e-f2e0-4f6f-b790-7b82cba47631'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.322614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] The token cache is being invalidate because project 8c905938a7c243e59de2059e8f5f62e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:33.324698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.326104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-82027d7e-f2e0-4f6f-b790-7b82cba47631 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.332324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 837/1673] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:32 2022] DELETE /identity/v3/projects/8c905938a7c243e59de2059e8f5f62e2 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:33.338938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399ca92-d957-4a49-b4d3-c589a40f785b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.339267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399ca92-d957-4a49-b4d3-c589a40f785b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.339581 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399ca92-d957-4a49-b4d3-c589a40f785b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.386405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1399ca92-d957-4a49-b4d3-c589a40f785b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:33.394267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1399ca92-d957-4a49-b4d3-c589a40f785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.409287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:33.414526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c3e4f-4ddc-44a3-b753-83130dd49f22 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.428062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974555ab-7f7e-4b4a-aba7-8791d405be56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.429393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974555ab-7f7e-4b4a-aba7-8791d405be56 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.429393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974555ab-7f7e-4b4a-aba7-8791d405be56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.429759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974555ab-7f7e-4b4a-aba7-8791d405be56 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.430387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974555ab-7f7e-4b4a-aba7-8791d405be56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.434470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 838/1674] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:33.435176 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974555ab-7f7e-4b4a-aba7-8791d405be56 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.446271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399ca92-d957-4a49-b4d3-c589a40f785b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.448843 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1399ca92-d957-4a49-b4d3-c589a40f785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.473422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 837/1675] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:33 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:33.479380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6772c-d543-4399-b0b0-efa3cc262571 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.479672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6772c-d543-4399-b0b0-efa3cc262571 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.480251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6772c-d543-4399-b0b0-efa3cc262571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.484690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399ca92-d957-4a49-b4d3-c589a40f785b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.491081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.492038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.493164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.493164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] PATH_INFO: `/v3/projects/84e65fadd26141bfa86232b347104628` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.494567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e65fadd26141bfa86232b347104628)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.496547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.529900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cdc6772c-d543-4399-b0b0-efa3cc262571 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:33.532552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cdc6772c-d543-4399-b0b0-efa3cc262571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.549315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 838/1676] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:33.551088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6772c-d543-4399-b0b0-efa3cc262571 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.553155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cdc6772c-d543-4399-b0b0-efa3cc262571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.568446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6772c-d543-4399-b0b0-efa3cc262571 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.585585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e65fadd26141bfa86232b347104628', 'request_id': 'req-b5477f22-ae8f-4473-82ef-69f4c7ea8425'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.607290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e65fadd26141bfa86232b347104628', 'request_id': 'req-b5477f22-ae8f-4473-82ef-69f4c7ea8425'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.633995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e65fadd26141bfa86232b347104628', 'request_id': 'req-b5477f22-ae8f-4473-82ef-69f4c7ea8425'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.681437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] The token cache is being invalidate because project 84e65fadd26141bfa86232b347104628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:33.681619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.686098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.698759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 839/1677] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:33 2022] DELETE /identity/v3/projects/84e65fadd26141bfa86232b347104628 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:33.699118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.704576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.711763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.713511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.713945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.714361 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] PATH_INFO: `/v3/projects/904fabbf81b748d98bbc0ff8b30f3f67` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.717631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=904fabbf81b748d98bbc0ff8b30f3f67)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.721005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.754297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477f22-ae8f-4473-82ef-69f4c7ea8425 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.774215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:33.775078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.775375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.775658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:33.779427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:33.784670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:33.822644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 840/1678] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:33.832027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:33.832566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904fabbf81b748d98bbc0ff8b30f3f67', 'request_id': 'req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.834110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:33.867507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904fabbf81b748d98bbc0ff8b30f3f67', 'request_id': 'req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.877907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e554b7-67a8-4135-a0cc-1dd17d01d93d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:33.885440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904fabbf81b748d98bbc0ff8b30f3f67', 'request_id': 'req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.918551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] The token cache is being invalidate because project 904fabbf81b748d98bbc0ff8b30f3f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:33.918857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.921653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d000accc-020b-4c77-a5f3-a1e3cb2dba8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:33.925255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 839/1679] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:33 2022] DELETE /identity/v3/projects/904fabbf81b748d98bbc0ff8b30f3f67 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:33.931370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177afe5-6cf4-412e-91fb-81b5b8c70cec None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:33.931370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177afe5-6cf4-412e-91fb-81b5b8c70cec None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:33.931370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177afe5-6cf4-412e-91fb-81b5b8c70cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.003218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d177afe5-6cf4-412e-91fb-81b5b8c70cec None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:34.008136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d177afe5-6cf4-412e-91fb-81b5b8c70cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:34.041992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.047067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.047357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.047825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.048625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:34.049155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.050907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.095163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 841/1680] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:34.095163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.098260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:34.146854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c423f5-ecfb-422e-ac79-7326d29a062c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.156311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.157015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.158121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.158415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.159167 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.160818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.185501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 840/1681] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:34.195747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216306b1-afba-475d-a0d1-123151479c8d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.196087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216306b1-afba-475d-a0d1-123151479c8d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.196380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216306b1-afba-475d-a0d1-123151479c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.210412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 842/1682] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:34.211627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.221888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:34.261340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-216306b1-afba-475d-a0d1-123151479c8d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:34.262326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-216306b1-afba-475d-a0d1-123151479c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:34.263554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb2850e-ca54-445e-996b-ff1c62243df2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.274124 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2150fc-a750-46a4-8988-ec593e128435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.274824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2150fc-a750-46a4-8988-ec593e128435 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.275114 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2150fc-a750-46a4-8988-ec593e128435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.275400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2150fc-a750-46a4-8988-ec593e128435 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.276128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2150fc-a750-46a4-8988-ec593e128435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.280736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2150fc-a750-46a4-8988-ec593e128435 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.284109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 841/1683] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:34.285541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef041b-5fa3-4765-8c3d-2d33601d5110 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.285834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef041b-5fa3-4765-8c3d-2d33601d5110 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.286116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef041b-5fa3-4765-8c3d-2d33601d5110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.310034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:34.319762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:34.326956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-71ef041b-5fa3-4765-8c3d-2d33601d5110 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:34.328579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-71ef041b-5fa3-4765-8c3d-2d33601d5110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:34.346032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 843/1684] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:34 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:34.347025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2150fc-a750-46a4-8988-ec593e128435 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.362037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 842/1685] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:34.367417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8fe00-83d2-4fcc-9a37-4600cf1b5f3e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.367713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8fe00-83d2-4fcc-9a37-4600cf1b5f3e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.367997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8fe00-83d2-4fcc-9a37-4600cf1b5f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.425486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-bdf8fe00-83d2-4fcc-9a37-4600cf1b5f3e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:34.428563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-bdf8fe00-83d2-4fcc-9a37-4600cf1b5f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:34.505028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2150fc-a750-46a4-8988-ec593e128435 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.524592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ae390-cd49-4768-92d2-616fcbc0e093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.526737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 843/1686] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:34.528050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8fe00-83d2-4fcc-9a37-4600cf1b5f3e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.528779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ae390-cd49-4768-92d2-616fcbc0e093 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.529068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ae390-cd49-4768-92d2-616fcbc0e093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.529388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ae390-cd49-4768-92d2-616fcbc0e093 admin admin] PATH_INFO: `/v3/projects/eaa003d534c34b2494fdbe9c893b5955/users/e616b4a0ffe84787bf4fa153d66daf20/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.540824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8fe00-83d2-4fcc-9a37-4600cf1b5f3e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.560560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9076c-ccd5-491e-a4dd-ce2daf1052a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.561305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9076c-ccd5-491e-a4dd-ce2daf1052a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.561585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9076c-ccd5-491e-a4dd-ce2daf1052a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.561858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9076c-ccd5-491e-a4dd-ce2daf1052a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.562564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9076c-ccd5-491e-a4dd-ce2daf1052a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.566237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ae390-cd49-4768-92d2-616fcbc0e093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa003d534c34b2494fdbe9c893b5955, user_id=e616b4a0ffe84787bf4fa153d66daf20, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.569263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ae390-cd49-4768-92d2-616fcbc0e093 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.570903 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9076c-ccd5-491e-a4dd-ce2daf1052a9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.604025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 844/1687] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:34 2022] PUT /identity/v3/projects/eaa003d534c34b2494fdbe9c893b5955/users/e616b4a0ffe84787bf4fa153d66daf20/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:34.619281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 844/1688] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:34 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:34.621874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ae390-cd49-4768-92d2-616fcbc0e093 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.623947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d035d-eb7b-4e4c-b93f-5bda55f7077e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.624406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d035d-eb7b-4e4c-b93f-5bda55f7077e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.624856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d035d-eb7b-4e4c-b93f-5bda55f7077e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.718141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-454d035d-eb7b-4e4c-b93f-5bda55f7077e None None] MFA Rules not processed for user `0cd8a309f6914d83b8e72bc60a5d7924`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:34.719260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-454d035d-eb7b-4e4c-b93f-5bda55f7077e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:34.791164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ae390-cd49-4768-92d2-616fcbc0e093 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.809757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.810456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.810748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.811473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.811790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:34.812102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.817817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.844454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 845/1689] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:34.848951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:34.867470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b255d6-058b-483e-94f5-7f0f80faf24f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:34.875513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b0d52-a572-4953-b332-b01157066061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:34.876208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b0d52-a572-4953-b332-b01157066061 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.876629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b0d52-a572-4953-b332-b01157066061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.876927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b0d52-a572-4953-b332-b01157066061 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.877853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b0d52-a572-4953-b332-b01157066061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:34.878175 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b0d52-a572-4953-b332-b01157066061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:34.880185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b0d52-a572-4953-b332-b01157066061 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:34.883096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 845/1690] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:34.886755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cef8da-a003-461f-8ef7-c7a8dae4973b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.887693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cef8da-a003-461f-8ef7-c7a8dae4973b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.887993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cef8da-a003-461f-8ef7-c7a8dae4973b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.923810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 846/1691] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:34.928729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5d1c1-613a-4433-ab0b-cd90667cb1ee None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:34.929543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5d1c1-613a-4433-ab0b-cd90667cb1ee None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:34.930039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5d1c1-613a-4433-ab0b-cd90667cb1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:34.979579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-46cef8da-a003-461f-8ef7-c7a8dae4973b None None] MFA Rules not processed for user `a132fd2fe24343239eafa8c46abaa2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:34.982187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-46cef8da-a003-461f-8ef7-c7a8dae4973b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:35.005748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c8e5d1c1-613a-4433-ab0b-cd90667cb1ee None None] MFA Rules not processed for user `e616b4a0ffe84787bf4fa153d66daf20`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:35.008306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c8e5d1c1-613a-4433-ab0b-cd90667cb1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:35.172831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:35.192807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 846/1692] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:35.194047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cef8da-a003-461f-8ef7-c7a8dae4973b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:35.210467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:35.244254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cef8da-a003-461f-8ef7-c7a8dae4973b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:35.258104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd526a71-e21c-4dd3-8d98-21ea47386229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:35.259170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd526a71-e21c-4dd3-8d98-21ea47386229 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:35.260104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 847/1693] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:35.260567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5d1c1-613a-4433-ab0b-cd90667cb1ee None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:35.261415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd526a71-e21c-4dd3-8d98-21ea47386229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:35.262046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd526a71-e21c-4dd3-8d98-21ea47386229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:35.265086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd526a71-e21c-4dd3-8d98-21ea47386229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:35.271113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd526a71-e21c-4dd3-8d98-21ea47386229 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:35.281012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5d1c1-613a-4433-ab0b-cd90667cb1ee None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:35.294738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9524c-3daa-44f6-a9cd-87d071495631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:35.295591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9524c-3daa-44f6-a9cd-87d071495631 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:35.295971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9524c-3daa-44f6-a9cd-87d071495631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:35.296460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9524c-3daa-44f6-a9cd-87d071495631 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:35.297654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9524c-3daa-44f6-a9cd-87d071495631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:35.299187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 847/1694] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:35.300811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3185-2f4c-40d9-8d03-5c1c8f547712 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:35.301102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3185-2f4c-40d9-8d03-5c1c8f547712 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:35.301407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af3185-2f4c-40d9-8d03-5c1c8f547712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:35.301912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9524c-3daa-44f6-a9cd-87d071495631 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:35.361421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 848/1695] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:35 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:35.367751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9524c-3daa-44f6-a9cd-87d071495631 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:35.368554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f4af3185-2f4c-40d9-8d03-5c1c8f547712 None None] MFA Rules not processed for user `0cd8a309f6914d83b8e72bc60a5d7924`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:35.373162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f4af3185-2f4c-40d9-8d03-5c1c8f547712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:35.583293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 848/1696] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:35.613086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a472ef-5b1a-4368-96a8-dbc309e59d00 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:35.613421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a472ef-5b1a-4368-96a8-dbc309e59d00 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:35.613732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a472ef-5b1a-4368-96a8-dbc309e59d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:35.627120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9524c-3daa-44f6-a9cd-87d071495631 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:35.646226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d16600-5366-4085-bb6e-dc012a0c8705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:35.648477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16600-5366-4085-bb6e-dc012a0c8705 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:35.650010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16600-5366-4085-bb6e-dc012a0c8705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:35.650481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16600-5366-4085-bb6e-dc012a0c8705 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:35.654323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d16600-5366-4085-bb6e-dc012a0c8705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:35.661622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d16600-5366-4085-bb6e-dc012a0c8705 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:35.670430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e8a472ef-5b1a-4368-96a8-dbc309e59d00 None None] MFA Rules not processed for user `a132fd2fe24343239eafa8c46abaa2ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:35.673878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e8a472ef-5b1a-4368-96a8-dbc309e59d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:35.736378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 849/1697] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:35 2022] POST /identity/v3/projects => generated 379 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:35.739972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d16600-5366-4085-bb6e-dc012a0c8705 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:35.817427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 849/1698] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:35.823359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a472ef-5b1a-4368-96a8-dbc309e59d00 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:35.863897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a472ef-5b1a-4368-96a8-dbc309e59d00 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:35.925441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d16600-5366-4085-bb6e-dc012a0c8705 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:35.932321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c46032-0a61-440d-bb79-2d4b146b0a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:35.933310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c46032-0a61-440d-bb79-2d4b146b0a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:35.935285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c46032-0a61-440d-bb79-2d4b146b0a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:35.935574 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c46032-0a61-440d-bb79-2d4b146b0a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:35.936326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c46032-0a61-440d-bb79-2d4b146b0a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:35.938514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c46032-0a61-440d-bb79-2d4b146b0a1e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.004736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 850/1699] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:35 2022] POST /identity/v3/projects => generated 387 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:36.006736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c46032-0a61-440d-bb79-2d4b146b0a1e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.021105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b0c46-546b-44b1-9c32-9457cfc0e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.025905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0c46-546b-44b1-9c32-9457cfc0e9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.026325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0c46-546b-44b1-9c32-9457cfc0e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.026653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b0c46-546b-44b1-9c32-9457cfc0e9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.027919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b0c46-546b-44b1-9c32-9457cfc0e9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.031579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b0c46-546b-44b1-9c32-9457cfc0e9ae admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.071434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 850/1700] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:36.076178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef2090-3dd9-4c59-bcc4-3ecbf78888be None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.077922 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef2090-3dd9-4c59-bcc4-3ecbf78888be None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.080346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef2090-3dd9-4c59-bcc4-3ecbf78888be None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.087559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 851/1701] 10.222.0.66 () {54 vars in 961 bytes} [Thu Jun 16 17:22:36 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:36.089222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef2090-3dd9-4c59-bcc4-3ecbf78888be None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.156218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:36.233570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c46032-0a61-440d-bb79-2d4b146b0a1e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:36.237650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:36.250072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b3fe9-b3a9-4578-b2a9-78c6e3a51d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.251656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b3fe9-b3a9-4578-b2a9-78c6e3a51d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.253393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b3fe9-b3a9-4578-b2a9-78c6e3a51d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.256585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b3fe9-b3a9-4578-b2a9-78c6e3a51d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.257460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b3fe9-b3a9-4578-b2a9-78c6e3a51d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.266142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b3fe9-b3a9-4578-b2a9-78c6e3a51d90 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.334739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef2090-3dd9-4c59-bcc4-3ecbf78888be None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:36.336814 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 851/1702] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:36 2022] POST /identity/v3/users => generated 296 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:36.351494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6980207-3025-4012-aa11-a3b94a5a4e11 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.352932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6980207-3025-4012-aa11-a3b94a5a4e11 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.354769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6980207-3025-4012-aa11-a3b94a5a4e11 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.365692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4830a4-9468-4e65-9441-c5793befbc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.366515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4830a4-9468-4e65-9441-c5793befbc01 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.367018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 852/1703] 10.222.0.66 () {54 vars in 961 bytes} [Thu Jun 16 17:22:36 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:36.369728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6980207-3025-4012-aa11-a3b94a5a4e11 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.370463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4830a4-9468-4e65-9441-c5793befbc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.371979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4830a4-9468-4e65-9441-c5793befbc01 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.377501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4830a4-9468-4e65-9441-c5793befbc01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.380888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4830a4-9468-4e65-9441-c5793befbc01 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.416917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6980207-3025-4012-aa11-a3b94a5a4e11 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:36.427275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2be387-86c7-46a3-b459-b4e0216c4f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.428870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2be387-86c7-46a3-b459-b4e0216c4f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.431243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2be387-86c7-46a3-b459-b4e0216c4f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.431854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2be387-86c7-46a3-b459-b4e0216c4f94 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.436382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2be387-86c7-46a3-b459-b4e0216c4f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.438492 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 852/1704] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:36 2022] POST /identity/v3/users => generated 290 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:36.439489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4830a4-9468-4e65-9441-c5793befbc01 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.440346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2be387-86c7-46a3-b459-b4e0216c4f94 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.460266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4830a4-9468-4e65-9441-c5793befbc01 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:36.470864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d10e4-c364-45ef-a1d2-56a1f087219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.474399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d10e4-c364-45ef-a1d2-56a1f087219d admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.475415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d10e4-c364-45ef-a1d2-56a1f087219d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.477971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d10e4-c364-45ef-a1d2-56a1f087219d admin admin] PATH_INFO: `/v3/projects/76cb621543b1486e95c0ec193887bf9d/users/3ab33bc83db049c29b4714fea2e07b0a/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.478823 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 853/1705] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:36 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:36.489795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f68029-c852-401d-aaa5-5b49951ffc5d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.489795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f68029-c852-401d-aaa5-5b49951ffc5d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.489795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f68029-c852-401d-aaa5-5b49951ffc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.516673 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d10e4-c364-45ef-a1d2-56a1f087219d admin admin] RBAC: Authorizing `identity:create_grant(project_id=76cb621543b1486e95c0ec193887bf9d, user_id=3ab33bc83db049c29b4714fea2e07b0a, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.520014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d10e4-c364-45ef-a1d2-56a1f087219d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.529544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e9f68029-c852-401d-aaa5-5b49951ffc5d None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:36.530980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9f68029-c852-401d-aaa5-5b49951ffc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:36.564898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 853/1706] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:36 2022] PUT /identity/v3/projects/76cb621543b1486e95c0ec193887bf9d/users/3ab33bc83db049c29b4714fea2e07b0a/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:36.566208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d10e4-c364-45ef-a1d2-56a1f087219d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.652230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 854/1707] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Jun 16 17:22:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:36.656353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31653b-5159-4f45-ac83-21defd207a28 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.657053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31653b-5159-4f45-ac83-21defd207a28 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.659315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca31653b-5159-4f45-ac83-21defd207a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.701516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ca31653b-5159-4f45-ac83-21defd207a28 None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:36.704862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ca31653b-5159-4f45-ac83-21defd207a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:36.747660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d10e4-c364-45ef-a1d2-56a1f087219d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:36.765298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f9fe-e694-4af8-83cc-332c4e6ace26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.766029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f9fe-e694-4af8-83cc-332c4e6ace26 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.766864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f9fe-e694-4af8-83cc-332c4e6ace26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.767081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384f9fe-e694-4af8-83cc-332c4e6ace26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.767832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384f9fe-e694-4af8-83cc-332c4e6ace26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.775292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384f9fe-e694-4af8-83cc-332c4e6ace26 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.812918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 854/1708] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:36.817617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f9fe-e694-4af8-83cc-332c4e6ace26 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.849140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384f9fe-e694-4af8-83cc-332c4e6ace26 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:36.874220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5efbf-e396-4af2-aa1b-18bac833972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.876851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5efbf-e396-4af2-aa1b-18bac833972b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.877981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5efbf-e396-4af2-aa1b-18bac833972b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.879045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5efbf-e396-4af2-aa1b-18bac833972b admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.880822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5efbf-e396-4af2-aa1b-18bac833972b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.885845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5efbf-e396-4af2-aa1b-18bac833972b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:36.892873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 855/1709] 10.222.0.66 () {58 vars in 1058 bytes} [Thu Jun 16 17:22:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:36.906007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31653b-5159-4f45-ac83-21defd207a28 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:36.934346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 855/1710] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:36.938412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36457b51-c5a0-41b1-9a1a-772c31fb8550 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.939459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36457b51-c5a0-41b1-9a1a-772c31fb8550 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.955015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36457b51-c5a0-41b1-9a1a-772c31fb8550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.961354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca31653b-5159-4f45-ac83-21defd207a28 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:36.973787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaab2d3-9bad-4e58-b6e0-3bb3ebb4069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:36.979196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaab2d3-9bad-4e58-b6e0-3bb3ebb4069c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:36.986315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaab2d3-9bad-4e58-b6e0-3bb3ebb4069c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:36.987411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaab2d3-9bad-4e58-b6e0-3bb3ebb4069c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:36.988791 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaab2d3-9bad-4e58-b6e0-3bb3ebb4069c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:36.994144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaab2d3-9bad-4e58-b6e0-3bb3ebb4069c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.058514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 856/1711] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:37.062254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaab2d3-9bad-4e58-b6e0-3bb3ebb4069c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.065058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-36457b51-c5a0-41b1-9a1a-772c31fb8550 None None] MFA Rules not processed for user `3ab33bc83db049c29b4714fea2e07b0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:37.066356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-36457b51-c5a0-41b1-9a1a-772c31fb8550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:37.110530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaab2d3-9bad-4e58-b6e0-3bb3ebb4069c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:37.129007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d8d46-a7c6-4c7f-9121-2f3227191ada service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:37.129961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d8d46-a7c6-4c7f-9121-2f3227191ada service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:37.131142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d8d46-a7c6-4c7f-9121-2f3227191ada service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:37.132291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667d8d46-a7c6-4c7f-9121-2f3227191ada service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:37.221098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:37.240282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 856/1712] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:37.241693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36457b51-c5a0-41b1-9a1a-772c31fb8550 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.255381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36457b51-c5a0-41b1-9a1a-772c31fb8550 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:37.261728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d0335-79e7-4be1-96d7-5dbe4d9bdd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:37.262783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d0335-79e7-4be1-96d7-5dbe4d9bdd22 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:37.263184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d0335-79e7-4be1-96d7-5dbe4d9bdd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:37.263571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74d0335-79e7-4be1-96d7-5dbe4d9bdd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:37.264620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d0335-79e7-4be1-96d7-5dbe4d9bdd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:37.266524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74d0335-79e7-4be1-96d7-5dbe4d9bdd22 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.286698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 857/1713] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:37.310498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d0335-79e7-4be1-96d7-5dbe4d9bdd22 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.323048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:37.344569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74d0335-79e7-4be1-96d7-5dbe4d9bdd22 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:37.356540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d6d3a-1eaa-49b3-91b4-1d251f7ee33c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:37.357457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d6d3a-1eaa-49b3-91b4-1d251f7ee33c service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:37.357860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d6d3a-1eaa-49b3-91b4-1d251f7ee33c service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:37.358252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d6d3a-1eaa-49b3-91b4-1d251f7ee33c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:37.429781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d8d46-a7c6-4c7f-9121-2f3227191ada service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:37.441042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667d8d46-a7c6-4c7f-9121-2f3227191ada service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.493138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 857/1714] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:37.498826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d8d46-a7c6-4c7f-9121-2f3227191ada service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.532592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667d8d46-a7c6-4c7f-9121-2f3227191ada service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:37.552482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b5ecd-cc76-459e-baa6-1b13f8f4da4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:37.555473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b5ecd-cc76-459e-baa6-1b13f8f4da4b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:37.556993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b5ecd-cc76-459e-baa6-1b13f8f4da4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:37.564698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b5ecd-cc76-459e-baa6-1b13f8f4da4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:37.568216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b5ecd-cc76-459e-baa6-1b13f8f4da4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:37.569954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b5ecd-cc76-459e-baa6-1b13f8f4da4b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.623024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 858/1715] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:37.624238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b5ecd-cc76-459e-baa6-1b13f8f4da4b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.649364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d6d3a-1eaa-49b3-91b4-1d251f7ee33c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:37.653866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d6d3a-1eaa-49b3-91b4-1d251f7ee33c service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.664631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b5ecd-cc76-459e-baa6-1b13f8f4da4b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:37.676392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71992847-1d7d-4b89-b843-6bd32c5e52c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:37.677515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71992847-1d7d-4b89-b843-6bd32c5e52c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:37.677899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71992847-1d7d-4b89-b843-6bd32c5e52c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:37.678297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71992847-1d7d-4b89-b843-6bd32c5e52c4 admin admin] PATH_INFO: `/v3/projects/26bcf5bb50b645fdb1842352b0774062/users/03f47c9b60bb4d949179c62ea70f4bda/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:37.679227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 858/1716] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:37.680253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d6d3a-1eaa-49b3-91b4-1d251f7ee33c service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.698847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d6d3a-1eaa-49b3-91b4-1d251f7ee33c service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:37.724623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83eddd-5865-4721-82ad-90064d480374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:37.726184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83eddd-5865-4721-82ad-90064d480374 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:37.726678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83eddd-5865-4721-82ad-90064d480374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:37.727639 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83eddd-5865-4721-82ad-90064d480374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:37.729117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83eddd-5865-4721-82ad-90064d480374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:37.732569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c83eddd-5865-4721-82ad-90064d480374 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.738986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71992847-1d7d-4b89-b843-6bd32c5e52c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bcf5bb50b645fdb1842352b0774062, user_id=03f47c9b60bb4d949179c62ea70f4bda, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:37.748946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71992847-1d7d-4b89-b843-6bd32c5e52c4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.784284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 859/1717] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:37 2022] PUT /identity/v3/projects/26bcf5bb50b645fdb1842352b0774062/users/03f47c9b60bb4d949179c62ea70f4bda/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:37.785832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71992847-1d7d-4b89-b843-6bd32c5e52c4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.828470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 859/1718] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:37 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:37.828470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83eddd-5865-4721-82ad-90064d480374 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.893618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c83eddd-5865-4721-82ad-90064d480374 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:37.898703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b664cac-eb5c-4962-8c32-adc02b896292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:37.899613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664cac-eb5c-4962-8c32-adc02b896292 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:37.899905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664cac-eb5c-4962-8c32-adc02b896292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:37.900185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b664cac-eb5c-4962-8c32-adc02b896292 admin admin] PATH_INFO: `/v3/projects/2fcce555e476489288a632478b45bddd/users/e4f9d0114da342c8b71842da45834f26/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:37.931048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b664cac-eb5c-4962-8c32-adc02b896292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fcce555e476489288a632478b45bddd, user_id=e4f9d0114da342c8b71842da45834f26, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:37.933706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b664cac-eb5c-4962-8c32-adc02b896292 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:37.956247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 860/1719] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:37 2022] PUT /identity/v3/projects/2fcce555e476489288a632478b45bddd/users/e4f9d0114da342c8b71842da45834f26/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:37.957230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b664cac-eb5c-4962-8c32-adc02b896292 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:37.959868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71992847-1d7d-4b89-b843-6bd32c5e52c4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:38.116587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe76dc1-f79e-4c22-a6cd-e965005fcab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:38.117613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe76dc1-f79e-4c22-a6cd-e965005fcab9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.117932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe76dc1-f79e-4c22-a6cd-e965005fcab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.118233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe76dc1-f79e-4c22-a6cd-e965005fcab9 admin admin] PATH_INFO: `/v3/projects/15b95a6aa1794c16afee59c287754fd2/users/c833e44e43d14f7c86ad0d42217c93ce/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.153933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe76dc1-f79e-4c22-a6cd-e965005fcab9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15b95a6aa1794c16afee59c287754fd2, user_id=c833e44e43d14f7c86ad0d42217c93ce, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:38.156604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe76dc1-f79e-4c22-a6cd-e965005fcab9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:38.161235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b664cac-eb5c-4962-8c32-adc02b896292 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:38.179013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01621fa-d1f1-40e0-8292-3002d6ca50a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:38.179803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 860/1720] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:37 2022] PUT /identity/v3/projects/15b95a6aa1794c16afee59c287754fd2/users/c833e44e43d14f7c86ad0d42217c93ce/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:38.181198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097b257-1cf9-4fc0-93d4-8f3c1a2f9a27 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.181543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097b257-1cf9-4fc0-93d4-8f3c1a2f9a27 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.181828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097b257-1cf9-4fc0-93d4-8f3c1a2f9a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.194502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01621fa-d1f1-40e0-8292-3002d6ca50a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.194852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01621fa-d1f1-40e0-8292-3002d6ca50a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.195211 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01621fa-d1f1-40e0-8292-3002d6ca50a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.196244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01621fa-d1f1-40e0-8292-3002d6ca50a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:38.199877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01621fa-d1f1-40e0-8292-3002d6ca50a5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:38.219436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:38.237697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 861/1721] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:38.238669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01621fa-d1f1-40e0-8292-3002d6ca50a5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:38.245730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5097b257-1cf9-4fc0-93d4-8f3c1a2f9a27 None None] MFA Rules not processed for user `1a3418bd5d7f430ca84efd5c83e38fed`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:38.246990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5097b257-1cf9-4fc0-93d4-8f3c1a2f9a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:38.253414 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:38.271992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01621fa-d1f1-40e0-8292-3002d6ca50a5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:38.463506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 861/1722] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:38.481354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb4a9c-64fe-4b30-b564-3d0bb7f2ead5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.484853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb4a9c-64fe-4b30-b564-3d0bb7f2ead5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.493406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb4a9c-64fe-4b30-b564-3d0bb7f2ead5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.499742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8b0b2-b546-49af-83a3-c9341f5606cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:38.503881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8b0b2-b546-49af-83a3-c9341f5606cf admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.503881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8b0b2-b546-49af-83a3-c9341f5606cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.503881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8b0b2-b546-49af-83a3-c9341f5606cf admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.506342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8b0b2-b546-49af-83a3-c9341f5606cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:38.508409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8b0b2-b546-49af-83a3-c9341f5606cf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:38.560033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 862/1723] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:38 2022] POST /identity/v3/users => generated 291 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:38.560033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c832c8d-c81f-4537-9a7a-d0860b2b77ec None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.560033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c832c8d-c81f-4537-9a7a-d0860b2b77ec None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.560033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c832c8d-c81f-4537-9a7a-d0860b2b77ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.562876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a7bb4a9c-64fe-4b30-b564-3d0bb7f2ead5 None None] MFA Rules not processed for user `e4f9d0114da342c8b71842da45834f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:38.564215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7bb4a9c-64fe-4b30-b564-3d0bb7f2ead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:38.638566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8c832c8d-c81f-4537-9a7a-d0860b2b77ec None None] MFA Rules not processed for user `c833e44e43d14f7c86ad0d42217c93ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:38.640337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8c832c8d-c81f-4537-9a7a-d0860b2b77ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:38.769534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 862/1724] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:38.772201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2a7cc-03b7-45ee-8efd-52fa66b1a168 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.772868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2a7cc-03b7-45ee-8efd-52fa66b1a168 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.773311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2a7cc-03b7-45ee-8efd-52fa66b1a168 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.775582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 863/1725] 10.222.0.66 () {54 vars in 912 bytes} [Thu Jun 16 17:22:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:38.776551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2a7cc-03b7-45ee-8efd-52fa66b1a168 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:38.818320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2a7cc-03b7-45ee-8efd-52fa66b1a168 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:38.827830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f16a2-1d9a-4fe5-bfe5-e69411b8345f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:38.829537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f16a2-1d9a-4fe5-bfe5-e69411b8345f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.830104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f16a2-1d9a-4fe5-bfe5-e69411b8345f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.830521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f16a2-1d9a-4fe5-bfe5-e69411b8345f admin admin] PATH_INFO: `/v3/projects/26bcf5bb50b645fdb1842352b0774062/users/03f47c9b60bb4d949179c62ea70f4bda/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.833292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 863/1726] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:38.834022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f16a2-1d9a-4fe5-bfe5-e69411b8345f admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bcf5bb50b645fdb1842352b0774062, user_id=03f47c9b60bb4d949179c62ea70f4bda, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:38.834990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d11ba-1890-4170-81be-e799011c87fb None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:38.835285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d11ba-1890-4170-81be-e799011c87fb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:38.835572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d11ba-1890-4170-81be-e799011c87fb None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:38.838078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426f16a2-1d9a-4fe5-bfe5-e69411b8345f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:38.851276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 864/1728] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:38 2022] PUT /identity/v3/projects/26bcf5bb50b645fdb1842352b0774062/users/03f47c9b60bb4d949179c62ea70f4bda/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:38.851826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 864/1728] 10.222.0.66 () {54 vars in 912 bytes} [Thu Jun 16 17:22:38 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:38.853042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f16a2-1d9a-4fe5-bfe5-e69411b8345f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:38.855280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d11ba-1890-4170-81be-e799011c87fb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.003237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f16a2-1d9a-4fe5-bfe5-e69411b8345f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.016214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d23b34-6236-4ea3-b472-1925920dabd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.016969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d23b34-6236-4ea3-b472-1925920dabd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.017295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d23b34-6236-4ea3-b472-1925920dabd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.017571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d23b34-6236-4ea3-b472-1925920dabd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.018325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d23b34-6236-4ea3-b472-1925920dabd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.019988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d23b34-6236-4ea3-b472-1925920dabd5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.037474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d11ba-1890-4170-81be-e799011c87fb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.058932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 865/1729] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:39.059475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888f5ee-499d-4bcf-b2bf-23e7b2d45739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.059997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0442fd-526d-45f2-a81b-80a72c1090bd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.060182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f5ee-499d-4bcf-b2bf-23e7b2d45739 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.060285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0442fd-526d-45f2-a81b-80a72c1090bd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.060473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f5ee-499d-4bcf-b2bf-23e7b2d45739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.060550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0442fd-526d-45f2-a81b-80a72c1090bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.061005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2888f5ee-499d-4bcf-b2bf-23e7b2d45739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.061836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888f5ee-499d-4bcf-b2bf-23e7b2d45739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.069366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2888f5ee-499d-4bcf-b2bf-23e7b2d45739 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.099942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0f0442fd-526d-45f2-a81b-80a72c1090bd None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:39.101349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f0442fd-526d-45f2-a81b-80a72c1090bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:39.125348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:39.131795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 865/1730] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:38 2022] POST /identity/v3/projects => generated 381 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:39.132013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888f5ee-499d-4bcf-b2bf-23e7b2d45739 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.155774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:39.274825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 866/1731] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:22:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:39.277903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e0c05-164e-4c68-9dc8-730cece9d6f5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.279013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e0c05-164e-4c68-9dc8-730cece9d6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.280803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e0c05-164e-4c68-9dc8-730cece9d6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.309801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2888f5ee-499d-4bcf-b2bf-23e7b2d45739 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.318670 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c39d3-6ff8-4bb8-85d9-ec0f6ad613c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.320093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c39d3-6ff8-4bb8-85d9-ec0f6ad613c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.320386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c39d3-6ff8-4bb8-85d9-ec0f6ad613c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.320855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c39d3-6ff8-4bb8-85d9-ec0f6ad613c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.321626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c39d3-6ff8-4bb8-85d9-ec0f6ad613c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.323963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c39d3-6ff8-4bb8-85d9-ec0f6ad613c7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.327758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f58e0c05-164e-4c68-9dc8-730cece9d6f5 None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:39.329140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f58e0c05-164e-4c68-9dc8-730cece9d6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:39.370828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 866/1732] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:39 2022] POST /identity/v3/projects => generated 393 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:39.372833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271339b6-59ce-48f3-9f14-8558a2c03ebf None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.373132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271339b6-59ce-48f3-9f14-8558a2c03ebf None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.373472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271339b6-59ce-48f3-9f14-8558a2c03ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.403764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-271339b6-59ce-48f3-9f14-8558a2c03ebf None None] MFA Rules not processed for user `03f47c9b60bb4d949179c62ea70f4bda`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:39.405000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-271339b6-59ce-48f3-9f14-8558a2c03ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:39.463228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 867/1733] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:22:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:39.464307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e0c05-164e-4c68-9dc8-730cece9d6f5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.482394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e0c05-164e-4c68-9dc8-730cece9d6f5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.526972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 867/1734] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:39.535349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271339b6-59ce-48f3-9f14-8558a2c03ebf None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.603697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.607691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.608073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.608357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] PATH_INFO: `/v3/projects/f3afc9b4f63a4595af85b9746cf1d24f/users/d02898c0baba4567ad3a6ac6ad77a0fa/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.632605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3afc9b4f63a4595af85b9746cf1d24f, user_id=d02898c0baba4567ad3a6ac6ad77a0fa, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.637522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.650175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271339b6-59ce-48f3-9f14-8558a2c03ebf None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.664169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 868/1735] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:39 2022] PUT /identity/v3/projects/f3afc9b4f63a4595af85b9746cf1d24f/users/d02898c0baba4567ad3a6ac6ad77a0fa/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:39.664169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.664630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:39.681377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:39.717335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:39.763081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c7bba-ae0e-4db7-bb79-d75e93eb2f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.764776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c7bba-ae0e-4db7-bb79-d75e93eb2f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.766721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c7bba-ae0e-4db7-bb79-d75e93eb2f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.767145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c7bba-ae0e-4db7-bb79-d75e93eb2f33 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.767984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c7bba-ae0e-4db7-bb79-d75e93eb2f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.770290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c7bba-ae0e-4db7-bb79-d75e93eb2f33 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.770463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea8b386-b00d-4724-a36f-417baf193b86 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.778968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2553f33-720f-41c9-a1c8-269d0943de3a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.779661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2553f33-720f-41c9-a1c8-269d0943de3a service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.779956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2553f33-720f-41c9-a1c8-269d0943de3a service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.780244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2553f33-720f-41c9-a1c8-269d0943de3a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.815724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 868/1736] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:39 2022] POST /identity/v3/users => generated 297 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:39.818597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709c49-1e2d-478f-ade1-b31374355a76 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.820455 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709c49-1e2d-478f-ade1-b31374355a76 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.821263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a709c49-1e2d-478f-ade1-b31374355a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.890155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9a709c49-1e2d-478f-ade1-b31374355a76 None None] MFA Rules not processed for user `68da2fd51cc84b2b8d641e1e6ac17c72`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:39.891118 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9a709c49-1e2d-478f-ade1-b31374355a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:39.933990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2553f33-720f-41c9-a1c8-269d0943de3a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.935997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2553f33-720f-41c9-a1c8-269d0943de3a service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:39.949520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 869/1737] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:39.950610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2553f33-720f-41c9-a1c8-269d0943de3a service glance] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:39.964136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2553f33-720f-41c9-a1c8-269d0943de3a service glance] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:39.971798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:39.972932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:39.973383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:39.973770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:39.974588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:39.977674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.019869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 870/1738] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:39 2022] POST /identity/v3/users => generated 303 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:40.019869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.021060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:40.048113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef88aae-5ce5-4d75-80e4-6da0c6cd9560 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.052878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af75ae-2619-43a7-a304-2ad37d68b35b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.053796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af75ae-2619-43a7-a304-2ad37d68b35b service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.054323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af75ae-2619-43a7-a304-2ad37d68b35b service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.054703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af75ae-2619-43a7-a304-2ad37d68b35b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.067597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 869/1739] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:40.067597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709c49-1e2d-478f-ade1-b31374355a76 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.071327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9a709c49-1e2d-478f-ade1-b31374355a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:40.077968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af75ae-2619-43a7-a304-2ad37d68b35b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.079682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af75ae-2619-43a7-a304-2ad37d68b35b service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.109556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 871/1740] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:40.113332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af75ae-2619-43a7-a304-2ad37d68b35b service glance] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.150440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af75ae-2619-43a7-a304-2ad37d68b35b service glance] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.190778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee232993-1c53-45b4-b816-a6c60c73658e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.191704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee232993-1c53-45b4-b816-a6c60c73658e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.192174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee232993-1c53-45b4-b816-a6c60c73658e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.192613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee232993-1c53-45b4-b816-a6c60c73658e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.193584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee232993-1c53-45b4-b816-a6c60c73658e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.199012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee232993-1c53-45b4-b816-a6c60c73658e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.199975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a709c49-1e2d-478f-ade1-b31374355a76 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.211418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c9459-c20e-4525-971e-f26c89dd90b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.213147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c9459-c20e-4525-971e-f26c89dd90b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.214867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 872/1741] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:40.215962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee232993-1c53-45b4-b816-a6c60c73658e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.218755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c9459-c20e-4525-971e-f26c89dd90b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.221012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c9459-c20e-4525-971e-f26c89dd90b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.235136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee232993-1c53-45b4-b816-a6c60c73658e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.249604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4e260-21d0-47f6-bf2e-045dd9c0f589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.251995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4e260-21d0-47f6-bf2e-045dd9c0f589 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.252508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4e260-21d0-47f6-bf2e-045dd9c0f589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.253670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f4e260-21d0-47f6-bf2e-045dd9c0f589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.256754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4e260-21d0-47f6-bf2e-045dd9c0f589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.260422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f4e260-21d0-47f6-bf2e-045dd9c0f589 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.284344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 873/1742] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:40.285524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4e260-21d0-47f6-bf2e-045dd9c0f589 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.311828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f4e260-21d0-47f6-bf2e-045dd9c0f589 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.322380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c559b7-b52b-4ff3-9d14-823004b9e4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.323080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c559b7-b52b-4ff3-9d14-823004b9e4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.323371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c559b7-b52b-4ff3-9d14-823004b9e4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.323642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c559b7-b52b-4ff3-9d14-823004b9e4c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.324387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c559b7-b52b-4ff3-9d14-823004b9e4c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.326047 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c559b7-b52b-4ff3-9d14-823004b9e4c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.338868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:40.354624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 874/1743] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:40.354624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c559b7-b52b-4ff3-9d14-823004b9e4c9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.362523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c9459-c20e-4525-971e-f26c89dd90b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.364148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c9459-c20e-4525-971e-f26c89dd90b8 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.378830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c559b7-b52b-4ff3-9d14-823004b9e4c9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.388857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 870/1744] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:40.390845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c9459-c20e-4525-971e-f26c89dd90b8 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.392101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5ace8-6556-4c26-952f-c7e2f31418df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.393034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ace8-6556-4c26-952f-c7e2f31418df admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.394520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ace8-6556-4c26-952f-c7e2f31418df admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.394878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab5ace8-6556-4c26-952f-c7e2f31418df admin admin] PATH_INFO: `/v3/projects/f3afc9b4f63a4595af85b9746cf1d24f/users/d02898c0baba4567ad3a6ac6ad77a0fa/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.402680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5ace8-6556-4c26-952f-c7e2f31418df admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3afc9b4f63a4595af85b9746cf1d24f, user_id=d02898c0baba4567ad3a6ac6ad77a0fa, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.405625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab5ace8-6556-4c26-952f-c7e2f31418df admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.414997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c9459-c20e-4525-971e-f26c89dd90b8 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.425410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b6dddc-066b-463a-bda1-a073d7a86c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.426584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6dddc-066b-463a-bda1-a073d7a86c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.426869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6dddc-066b-463a-bda1-a073d7a86c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.427147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6dddc-066b-463a-bda1-a073d7a86c7c admin admin] PATH_INFO: `/v3/projects/59bf1fcbe8624af68647407d0fca1bcc/users/a5d45f4527d4486d902c400ee6895d51/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.436893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 875/1745] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:40 2022] PUT /identity/v3/projects/f3afc9b4f63a4595af85b9746cf1d24f/users/d02898c0baba4567ad3a6ac6ad77a0fa/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:40.436893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5ace8-6556-4c26-952f-c7e2f31418df admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.488437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6dddc-066b-463a-bda1-a073d7a86c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bf1fcbe8624af68647407d0fca1bcc, user_id=a5d45f4527d4486d902c400ee6895d51, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.491199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b6dddc-066b-463a-bda1-a073d7a86c7c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.541010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 871/1746] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:40 2022] PUT /identity/v3/projects/59bf1fcbe8624af68647407d0fca1bcc/users/a5d45f4527d4486d902c400ee6895d51/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:40.543320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3bb5-2b15-40b9-bbee-5db28281d35c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.545026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3bb5-2b15-40b9-bbee-5db28281d35c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.545374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3bb5-2b15-40b9-bbee-5db28281d35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.602594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:40.607771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-283a3bb5-2b15-40b9-bbee-5db28281d35c None None] MFA Rules not processed for user `d02898c0baba4567ad3a6ac6ad77a0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:40.609048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-283a3bb5-2b15-40b9-bbee-5db28281d35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:40.639320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab5ace8-6556-4c26-952f-c7e2f31418df admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.725077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e4525-f75b-4ad4-b671-4bc68dab49eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.726270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e4525-f75b-4ad4-b671-4bc68dab49eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.726551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e4525-f75b-4ad4-b671-4bc68dab49eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.727015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e4525-f75b-4ad4-b671-4bc68dab49eb admin admin] PATH_INFO: `/v3/projects/c69880e6ee844f8da325ab4902350dc2/users/7c2d66447744453fbe6a5dd8ed88fb9a/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.742166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 872/1747] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:40 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:40.743329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a3bb5-2b15-40b9-bbee-5db28281d35c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.752646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e4525-f75b-4ad4-b671-4bc68dab49eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69880e6ee844f8da325ab4902350dc2, user_id=7c2d66447744453fbe6a5dd8ed88fb9a, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.756229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a3bb5-2b15-40b9-bbee-5db28281d35c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.756373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e4525-f75b-4ad4-b671-4bc68dab49eb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.761836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.763013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.763431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.763718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] PATH_INFO: `/v3/users/e616b4a0ffe84787bf4fa153d66daf20` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.765669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e616b4a0ffe84787bf4fa153d66daf20)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.767926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.773146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 876/1748] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:40 2022] PUT /identity/v3/projects/c69880e6ee844f8da325ab4902350dc2/users/7c2d66447744453fbe6a5dd8ed88fb9a/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:40.774653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e4525-f75b-4ad4-b671-4bc68dab49eb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.833121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e616b4a0ffe84787bf4fa153d66daf20'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:40.852554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e4525-f75b-4ad4-b671-4bc68dab49eb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.852814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e616b4a0ffe84787bf4fa153d66daf20'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:40.853628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e616b4a0ffe84787bf4fa153d66daf20'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:40.858294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db3017-d05c-494e-b5a9-3e951b8225c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.859204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3017-d05c-494e-b5a9-3e951b8225c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.859591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3017-d05c-494e-b5a9-3e951b8225c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.859986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db3017-d05c-494e-b5a9-3e951b8225c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.860866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db3017-d05c-494e-b5a9-3e951b8225c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.862759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db3017-d05c-494e-b5a9-3e951b8225c8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:40.864623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e616b4a0ffe84787bf4fa153d66daf20'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:40.874738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2ad270d7-e741-46fe-888b-a5da850f2782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e616b4a0ffe84787bf4fa153d66daf20'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:40.877780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 877/1749] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:40.882617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059c21e-4250-4f62-99da-b873dd1a3690 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.883104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059c21e-4250-4f62-99da-b873dd1a3690 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.883495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059c21e-4250-4f62-99da-b873dd1a3690 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.884864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 873/1750] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:40 2022] DELETE /identity/v3/users/e616b4a0ffe84787bf4fa153d66daf20 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:40.886515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30801e1-c075-4ec4-b41e-abd70f99e5c4 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.887046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30801e1-c075-4ec4-b41e-abd70f99e5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.887792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30801e1-c075-4ec4-b41e-abd70f99e5c4 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.891187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 878/1751] 10.222.0.66 () {54 vars in 912 bytes} [Thu Jun 16 17:22:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:40.891187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059c21e-4250-4f62-99da-b873dd1a3690 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.892583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e059c21e-4250-4f62-99da-b873dd1a3690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:40.893370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 874/1752] 10.222.0.66 () {54 vars in 912 bytes} [Thu Jun 16 17:22:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:40.895133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30801e1-c075-4ec4-b41e-abd70f99e5c4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:40.901845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a30801e1-c075-4ec4-b41e-abd70f99e5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:40.935287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:40.972301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059c21e-4250-4f62-99da-b873dd1a3690 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.975920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1196f-5a2f-4454-9cb9-64addc4180fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.976571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1196f-5a2f-4454-9cb9-64addc4180fa service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.976902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1196f-5a2f-4454-9cb9-64addc4180fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.977166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1196f-5a2f-4454-9cb9-64addc4180fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.979355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44e1196f-5a2f-4454-9cb9-64addc4180fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:40.987781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30801e1-c075-4ec4-b41e-abd70f99e5c4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:40.994239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:40.994944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:40.995277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:40.995617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:40.996486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:40.998343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:41.011092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 875/1753] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:41.012109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:41.015514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.022243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:41.029997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e63f6f-5ca2-4c21-ada6-5749e1c8f73e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:41.035767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9928ddd-4ddb-4eba-b997-d6a3c430f390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:41.036683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9928ddd-4ddb-4eba-b997-d6a3c430f390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.036964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9928ddd-4ddb-4eba-b997-d6a3c430f390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.037265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9928ddd-4ddb-4eba-b997-d6a3c430f390 admin admin] PATH_INFO: `/v3/projects/59bf1fcbe8624af68647407d0fca1bcc/users/a5d45f4527d4486d902c400ee6895d51/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.041925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9928ddd-4ddb-4eba-b997-d6a3c430f390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bf1fcbe8624af68647407d0fca1bcc, user_id=a5d45f4527d4486d902c400ee6895d51, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:41.044547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9928ddd-4ddb-4eba-b997-d6a3c430f390 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:41.057274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 876/1754] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:41 2022] PUT /identity/v3/projects/59bf1fcbe8624af68647407d0fca1bcc/users/a5d45f4527d4486d902c400ee6895d51/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:41.059445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b1389-3060-49eb-8f49-483e67620e76 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.059734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b1389-3060-49eb-8f49-483e67620e76 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.060632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b1389-3060-49eb-8f49-483e67620e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.085913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1196f-5a2f-4454-9cb9-64addc4180fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:41.087787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1196f-5a2f-4454-9cb9-64addc4180fa service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:41.088154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d42b1389-3060-49eb-8f49-483e67620e76 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:41.089412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d42b1389-3060-49eb-8f49-483e67620e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.100077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 879/1755] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:40 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:41.101268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e45022-116d-487d-ab01-eb3c447bffb6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.101558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e45022-116d-487d-ab01-eb3c447bffb6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.102098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e45022-116d-487d-ab01-eb3c447bffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.175073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 877/1756] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:41.176741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc2b5a-fc7e-49c6-8734-2a951cca4dec None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.177038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc2b5a-fc7e-49c6-8734-2a951cca4dec None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.177475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc2b5a-fc7e-49c6-8734-2a951cca4dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.200423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-90e45022-116d-487d-ab01-eb3c447bffb6 None None] MFA Rules not processed for user `72fb6996e1974060b162070f8da365aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:41.202319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a8cc2b5a-fc7e-49c6-8734-2a951cca4dec None None] MFA Rules not processed for user `72fb6996e1974060b162070f8da365aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:41.205222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a8cc2b5a-fc7e-49c6-8734-2a951cca4dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.205832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-90e45022-116d-487d-ab01-eb3c447bffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.272385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 878/1757] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:22:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:41.274583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc2b5a-fc7e-49c6-8734-2a951cca4dec None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:41.291268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc2b5a-fc7e-49c6-8734-2a951cca4dec None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:41.291922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 880/1758] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:22:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:41.297320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:41.298568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.298990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e698d-e8a7-4b64-8548-aa4cb76c9c28 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.299102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.299304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] PATH_INFO: `/v3/projects/c69880e6ee844f8da325ab4902350dc2/users/7c2d66447744453fbe6a5dd8ed88fb9a/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.299692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e698d-e8a7-4b64-8548-aa4cb76c9c28 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.301492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e698d-e8a7-4b64-8548-aa4cb76c9c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.301618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c69880e6ee844f8da325ab4902350dc2, user_id=7c2d66447744453fbe6a5dd8ed88fb9a, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:41.304449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:41.319527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 879/1759] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:41 2022] PUT /identity/v3/projects/c69880e6ee844f8da325ab4902350dc2/users/7c2d66447744453fbe6a5dd8ed88fb9a/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:41.320965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:41.323261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.345861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7c7e698d-e8a7-4b64-8548-aa4cb76c9c28 None None] MFA Rules not processed for user `a5d45f4527d4486d902c400ee6895d51`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:41.347441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7c7e698d-e8a7-4b64-8548-aa4cb76c9c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.397594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:41.410163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f860c168-223f-4dee-90bd-0a0436464d85 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:41.414363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:41.415678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.415678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.416064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.418307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.469089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 881/1760] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:41 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:41.483551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e698d-e8a7-4b64-8548-aa4cb76c9c28 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:41.484240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7c7e698d-e8a7-4b64-8548-aa4cb76c9c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.513734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:41.515409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:41.527663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 880/1761] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:41 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:41.528551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:41.531326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.590499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:41.604419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e698d-e8a7-4b64-8548-aa4cb76c9c28 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:41.622521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2850c-68b6-44b5-8a96-fa640a6e4519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:41.625256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2850c-68b6-44b5-8a96-fa640a6e4519 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.628278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2850c-68b6-44b5-8a96-fa640a6e4519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.630082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2850c-68b6-44b5-8a96-fa640a6e4519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.632180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1963cc-eefc-4456-aebd-487a4d5e57c2 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:41.637833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af42c4e-6891-41dd-bd56-56412a98856c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:41.638613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af42c4e-6891-41dd-bd56-56412a98856c service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.638995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af42c4e-6891-41dd-bd56-56412a98856c service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.639451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af42c4e-6891-41dd-bd56-56412a98856c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.651199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af42c4e-6891-41dd-bd56-56412a98856c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:41.654855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af42c4e-6891-41dd-bd56-56412a98856c service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:41.676223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 881/1762] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:41.676223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af42c4e-6891-41dd-bd56-56412a98856c service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:41.685765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af42c4e-6891-41dd-bd56-56412a98856c service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:41.690292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507f61d-d554-42eb-b861-d78f04cd65bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:41.691057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507f61d-d554-42eb-b861-d78f04cd65bd service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.691437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507f61d-d554-42eb-b861-d78f04cd65bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.691826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507f61d-d554-42eb-b861-d78f04cd65bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.700598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507f61d-d554-42eb-b861-d78f04cd65bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:41.702416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507f61d-d554-42eb-b861-d78f04cd65bd service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:41.713733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 882/1763] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:41.723117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8b0ec-ee5f-43fc-a22b-417ce1c249b3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.723517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8b0ec-ee5f-43fc-a22b-417ce1c249b3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.723948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8b0ec-ee5f-43fc-a22b-417ce1c249b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.827683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6ea8b0ec-ee5f-43fc-a22b-417ce1c249b3 None None] MFA Rules not processed for user `7c2d66447744453fbe6a5dd8ed88fb9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:41.835546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6ea8b0ec-ee5f-43fc-a22b-417ce1c249b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:41.895833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2850c-68b6-44b5-8a96-fa640a6e4519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:41.897394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2850c-68b6-44b5-8a96-fa640a6e4519 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:41.925127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 882/1764] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:41.925988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2850c-68b6-44b5-8a96-fa640a6e4519 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:41.950295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2850c-68b6-44b5-8a96-fa640a6e4519 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:41.961729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109214-9e85-435c-a88f-66c0e241ab80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:41.962510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109214-9e85-435c-a88f-66c0e241ab80 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:41.962849 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109214-9e85-435c-a88f-66c0e241ab80 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:41.963125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109214-9e85-435c-a88f-66c0e241ab80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:41.985760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10109214-9e85-435c-a88f-66c0e241ab80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:41.992713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10109214-9e85-435c-a88f-66c0e241ab80 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:42.026063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 883/1765] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:41 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:42.035890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 883/1766] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:42.083227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109214-9e85-435c-a88f-66c0e241ab80 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:42.093332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109214-9e85-435c-a88f-66c0e241ab80 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:42.111870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:42.112562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.112850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.113233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.128065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:42.131538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:42.144582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 884/1767] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:42.295227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8b0ec-ee5f-43fc-a22b-417ce1c249b3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:42.316652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8b0ec-ee5f-43fc-a22b-417ce1c249b3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:42.331072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db1009d-a757-4bd5-ab0b-2d33fd39e69f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:42.334055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1009d-a757-4bd5-ab0b-2d33fd39e69f service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.335090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1009d-a757-4bd5-ab0b-2d33fd39e69f service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.335577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1009d-a757-4bd5-ab0b-2d33fd39e69f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.338946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5db1009d-a757-4bd5-ab0b-2d33fd39e69f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:42.362346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:42.463679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1009d-a757-4bd5-ab0b-2d33fd39e69f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:42.466127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db1009d-a757-4bd5-ab0b-2d33fd39e69f service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:42.534221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:42.534618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 884/1768] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:42.543475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:42.699629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ff91d-8db1-4418-aa8e-fa978156b5b9 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.700046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ff91d-8db1-4418-aa8e-fa978156b5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.700431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ff91d-8db1-4418-aa8e-fa978156b5b9 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.702626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 885/1769] 10.222.0.66 () {54 vars in 960 bytes} [Thu Jun 16 17:22:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:22:42.715174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef09a-16b4-4675-830c-9632b12c58c3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.715514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef09a-16b4-4675-830c-9632b12c58c3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.715799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef09a-16b4-4675-830c-9632b12c58c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.784116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2b0ef09a-16b4-4675-830c-9632b12c58c3 None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:42.794999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2b0ef09a-16b4-4675-830c-9632b12c58c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:42.858907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae9d58-d828-4755-b4e7-7087949ecf24 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:42.876537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:42.877925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:42.879109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:42.879608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:42.882380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:43.087226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 886/1770] 10.222.0.66 () {58 vars in 1057 bytes} [Thu Jun 16 17:22:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:22:43.089297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ef09a-16b4-4675-830c-9632b12c58c3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:43.134129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0ef09a-16b4-4675-830c-9632b12c58c3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:43.145717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:43.146674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:43.147144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:43.151963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:43.161118 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:43.298877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:43.302558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:43.331571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 885/1771] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 797 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:43.332927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:43.336305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:43.368085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:43.377259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:43.381850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7609b1f-05a6-4c95-b9de-248cc2f2797b service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:43.388499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:43.390501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:43.390951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:43.391453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:43.395738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:43.406804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:43.440350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 887/1772] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:43.445390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:43.454936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:43.512717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:43.515598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:43.529215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 886/1773] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:43.530851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:43.533852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:43.537831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51f1ed-b94d-42ee-8a73-f4cf5e01d6d3 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:43.560161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:43.561983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:43.562272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:43.562550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] PATH_INFO: `/v3/projects/eaa003d534c34b2494fdbe9c893b5955` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:43.573915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa003d534c34b2494fdbe9c893b5955)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:43.577855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2736a1c-a7fd-457e-a18c-160a18031ccf service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:43.589027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:43.595598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:43.595801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:43.596146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:43.601066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:43.615886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:43.681040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:43.689241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:43.754022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 887/1774] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:43.756725 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:43.811264 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55aedf-b004-4e8b-bf91-c78a4cc170ce service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:43.819105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:43.845576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa003d534c34b2494fdbe9c893b5955', 'request_id': 'req-7d3013b6-a934-4c02-a507-a097b9d0f1e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:43.882157 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa003d534c34b2494fdbe9c893b5955', 'request_id': 'req-7d3013b6-a934-4c02-a507-a097b9d0f1e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:43.917103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa003d534c34b2494fdbe9c893b5955', 'request_id': 'req-7d3013b6-a934-4c02-a507-a097b9d0f1e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:43.982836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] The token cache is being invalidate because project eaa003d534c34b2494fdbe9c893b5955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:43.983136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:43.988640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:43.994966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 888/1775] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:43 2022] DELETE /identity/v3/projects/eaa003d534c34b2494fdbe9c893b5955 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:44.002289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:44.010597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:44.164357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb72fe-1668-4990-bf1d-0baef6c4369d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:44.171198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb72fe-1668-4990-bf1d-0baef6c4369d service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:44.178219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb72fe-1668-4990-bf1d-0baef6c4369d service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:44.178983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb72fe-1668-4990-bf1d-0baef6c4369d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:44.186539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f4cb72fe-1668-4990-bf1d-0baef6c4369d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:44.247227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb72fe-1668-4990-bf1d-0baef6c4369d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:44.249320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb72fe-1668-4990-bf1d-0baef6c4369d service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:44.315098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 888/1776] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:44.346894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3013b6-a934-4c02-a507-a097b9d0f1e2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:44.362759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:44.363672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:44.363953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:44.371551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] PATH_INFO: `/v3/users/03f47c9b60bb4d949179c62ea70f4bda` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:44.376500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f47c9b60bb4d949179c62ea70f4bda)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:44.383539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:44.452294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae9a50-774c-48e4-a634-6e447465ebb0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:44.452804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae9a50-774c-48e4-a634-6e447465ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:44.453317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae9a50-774c-48e4-a634-6e447465ebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:44.547323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-83ae9a50-774c-48e4-a634-6e447465ebb0 None None] MFA Rules not processed for user `0cd8a309f6914d83b8e72bc60a5d7924`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:44.552750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-83ae9a50-774c-48e4-a634-6e447465ebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:44.697754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f47c9b60bb4d949179c62ea70f4bda'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:44.730376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 889/1777] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:44.781804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f47c9b60bb4d949179c62ea70f4bda'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:44.787246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f47c9b60bb4d949179c62ea70f4bda'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:44.835828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f47c9b60bb4d949179c62ea70f4bda'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:44.870630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-2be96e32-f088-46e0-a25c-73b5a64bcdfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f47c9b60bb4d949179c62ea70f4bda'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:44.923571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 889/1778] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:44 2022] DELETE /identity/v3/users/03f47c9b60bb4d949179c62ea70f4bda => generated 0 bytes in 905 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:47.874420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae9a50-774c-48e4-a634-6e447465ebb0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:47.927705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae9a50-774c-48e4-a634-6e447465ebb0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:48.025993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:48.172426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:48.173719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:48.176144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:48.176742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] PATH_INFO: `/v3/projects/26bcf5bb50b645fdb1842352b0774062` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:48.181544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26bcf5bb50b645fdb1842352b0774062)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:48.184235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:48.310195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26bcf5bb50b645fdb1842352b0774062', 'request_id': 'req-f15964d5-449a-4f18-966b-cb77ea490457'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:48.328067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26bcf5bb50b645fdb1842352b0774062', 'request_id': 'req-f15964d5-449a-4f18-966b-cb77ea490457'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:48.347225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26bcf5bb50b645fdb1842352b0774062', 'request_id': 'req-f15964d5-449a-4f18-966b-cb77ea490457'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:48.367921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] The token cache is being invalidate because project 26bcf5bb50b645fdb1842352b0774062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:48.369633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:48.370591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f15964d5-449a-4f18-966b-cb77ea490457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:48.376993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 890/1779] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:47 2022] DELETE /identity/v3/projects/26bcf5bb50b645fdb1842352b0774062 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:48.385850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70181c0-c0dd-412f-9c6b-c458fe0cdef0 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:48.386697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70181c0-c0dd-412f-9c6b-c458fe0cdef0 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:48.387159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70181c0-c0dd-412f-9c6b-c458fe0cdef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:48.488261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d70181c0-c0dd-412f-9c6b-c458fe0cdef0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:48.489901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d70181c0-c0dd-412f-9c6b-c458fe0cdef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:48.781467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:48.847241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 890/1780] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:48.862736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fac7c-33a2-4376-980a-8cdc5a0547f9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:48.863594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fac7c-33a2-4376-980a-8cdc5a0547f9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:48.863594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fac7c-33a2-4376-980a-8cdc5a0547f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:48.903239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ba4fac7c-33a2-4376-980a-8cdc5a0547f9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:48.904472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba4fac7c-33a2-4376-980a-8cdc5a0547f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:48.938554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 891/1781] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:48.942790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70181c0-c0dd-412f-9c6b-c458fe0cdef0 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:48.956773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70181c0-c0dd-412f-9c6b-c458fe0cdef0 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:48.963074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:48.964438 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:48.965005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:48.965559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:48.966734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:48.967224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:48.969560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:49.011187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fac7c-33a2-4376-980a-8cdc5a0547f9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:49.013364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 891/1782] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:49.030787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fac7c-33a2-4376-980a-8cdc5a0547f9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:49.042465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985be1f2-0c37-4051-95c2-617e1038eb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:49.043565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be1f2-0c37-4051-95c2-617e1038eb96 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.044231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be1f2-0c37-4051-95c2-617e1038eb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.045337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be1f2-0c37-4051-95c2-617e1038eb96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.047708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985be1f2-0c37-4051-95c2-617e1038eb96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:49.049917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985be1f2-0c37-4051-95c2-617e1038eb96 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:49.116568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 892/1783] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:49 2022] POST /identity/v3/projects => generated 407 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:22:49.121212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:49.139838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff92896-e716-43da-86b5-f7bf9c92e350 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:49.244791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c186f-30cd-4a3a-a975-680e2b577c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:49.246175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c186f-30cd-4a3a-a975-680e2b577c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.246616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c186f-30cd-4a3a-a975-680e2b577c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.247029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c186f-30cd-4a3a-a975-680e2b577c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.248078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c186f-30cd-4a3a-a975-680e2b577c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:49.252515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3c186f-30cd-4a3a-a975-680e2b577c02 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:49.295323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 892/1784] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:49 2022] POST /identity/v3/users => generated 304 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:49.296912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985be1f2-0c37-4051-95c2-617e1038eb96 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:49.309431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985be1f2-0c37-4051-95c2-617e1038eb96 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:49.314097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a76ad-cb5f-459e-ba81-7df91f8fc087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:49.315497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a76ad-cb5f-459e-ba81-7df91f8fc087 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.315976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a76ad-cb5f-459e-ba81-7df91f8fc087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.316446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a76ad-cb5f-459e-ba81-7df91f8fc087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.317459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a76ad-cb5f-459e-ba81-7df91f8fc087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:49.319292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a76ad-cb5f-459e-ba81-7df91f8fc087 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:49.337834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 893/1785] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:49.342043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c186f-30cd-4a3a-a975-680e2b577c02 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:49.371610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c186f-30cd-4a3a-a975-680e2b577c02 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:49.391057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efeba2-a253-45fd-ba1c-c26f6b640a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:49.392447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efeba2-a253-45fd-ba1c-c26f6b640a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.392931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efeba2-a253-45fd-ba1c-c26f6b640a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.393450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efeba2-a253-45fd-ba1c-c26f6b640a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.394713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efeba2-a253-45fd-ba1c-c26f6b640a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:49.396641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efeba2-a253-45fd-ba1c-c26f6b640a37 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:49.397900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe4156-1f19-4160-94d0-a948c09ddc2e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.398192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe4156-1f19-4160-94d0-a948c09ddc2e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.398473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe4156-1f19-4160-94d0-a948c09ddc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.418949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 893/1786] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:49.419391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dabe4156-1f19-4160-94d0-a948c09ddc2e None None] MFA Rules not processed for user `68da2fd51cc84b2b8d641e1e6ac17c72`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:49.420865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dabe4156-1f19-4160-94d0-a948c09ddc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:49.424286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efeba2-a253-45fd-ba1c-c26f6b640a37 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:49.444610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efeba2-a253-45fd-ba1c-c26f6b640a37 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:49.451572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e5b32-4792-44ae-a750-7605f65bf31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:49.453378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e5b32-4792-44ae-a750-7605f65bf31b admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.453909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e5b32-4792-44ae-a750-7605f65bf31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.454554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e5b32-4792-44ae-a750-7605f65bf31b admin admin] PATH_INFO: `/v3/projects/16fe1da4ddb845caa8fab0c2b7c46adc/users/3333fc6138a746fda98c85441e876ccf/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.476372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e5b32-4792-44ae-a750-7605f65bf31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16fe1da4ddb845caa8fab0c2b7c46adc, user_id=3333fc6138a746fda98c85441e876ccf, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:49.479512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e5b32-4792-44ae-a750-7605f65bf31b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:49.504901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 894/1787] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:49 2022] PUT /identity/v3/projects/16fe1da4ddb845caa8fab0c2b7c46adc/users/3333fc6138a746fda98c85441e876ccf/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:22:49.517541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39963cc-fab0-4a17-80e5-c188dd51a9c6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.517992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39963cc-fab0-4a17-80e5-c188dd51a9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.518404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39963cc-fab0-4a17-80e5-c188dd51a9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.556257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 894/1788] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:49.573667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b39963cc-fab0-4a17-80e5-c188dd51a9c6 None None] MFA Rules not processed for user `3333fc6138a746fda98c85441e876ccf`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:49.575058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b39963cc-fab0-4a17-80e5-c188dd51a9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:49.623406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe4156-1f19-4160-94d0-a948c09ddc2e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:49.625065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dabe4156-1f19-4160-94d0-a948c09ddc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:49.643963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe4156-1f19-4160-94d0-a948c09ddc2e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:49.659481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 895/1789] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:49.663786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39963cc-fab0-4a17-80e5-c188dd51a9c6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:49.667066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:49.697055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:49.729079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:49.730004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.730592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.730864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] PATH_INFO: `/v3/users/3ab33bc83db049c29b4714fea2e07b0a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.733876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab33bc83db049c29b4714fea2e07b0a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:49.735994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:49.782681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39963cc-fab0-4a17-80e5-c188dd51a9c6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:49.794001 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:49.794766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.795050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.795436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.796167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:49.800528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:49.822360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab33bc83db049c29b4714fea2e07b0a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:49.823283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab33bc83db049c29b4714fea2e07b0a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:49.824208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab33bc83db049c29b4714fea2e07b0a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:49.851041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab33bc83db049c29b4714fea2e07b0a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:49.857271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 896/1790] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:49 2022] POST /identity/v3/projects => generated 407 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:22:49.859728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:49.863921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:49.866180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f90b42-dcac-4279-8a13-4e6411e4c0eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab33bc83db049c29b4714fea2e07b0a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:49.901632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 895/1791] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:49 2022] DELETE /identity/v3/users/3ab33bc83db049c29b4714fea2e07b0a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:49.903296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99143926-445f-4067-9133-5f1f6f5656c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:49.904165 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99143926-445f-4067-9133-5f1f6f5656c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:49.904725 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99143926-445f-4067-9133-5f1f6f5656c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:49.927803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-99143926-445f-4067-9133-5f1f6f5656c8 None None] MFA Rules not processed for user `1a3418bd5d7f430ca84efd5c83e38fed`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:49.929213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99143926-445f-4067-9133-5f1f6f5656c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:49.996213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db103d6-9473-433c-9ce2-1c3c92a2c773 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:50.005622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 896/1792] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:50.007257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639377c-8227-4c37-8562-635001a4d325 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.007572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639377c-8227-4c37-8562-635001a4d325 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.007847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639377c-8227-4c37-8562-635001a4d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.037977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6639377c-8227-4c37-8562-635001a4d325 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:50.039592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6639377c-8227-4c37-8562-635001a4d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:50.096603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c34223-f0d6-4fe0-a6e8-f4fd7dd5388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:50.097465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34223-f0d6-4fe0-a6e8-f4fd7dd5388d admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.097819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34223-f0d6-4fe0-a6e8-f4fd7dd5388d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.098292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c34223-f0d6-4fe0-a6e8-f4fd7dd5388d admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.099221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c34223-f0d6-4fe0-a6e8-f4fd7dd5388d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:50.101431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c34223-f0d6-4fe0-a6e8-f4fd7dd5388d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:50.115291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:50.137350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 897/1793] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:49 2022] POST /identity/v3/users => generated 310 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:50.141457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c34223-f0d6-4fe0-a6e8-f4fd7dd5388d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:50.147440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 897/1794] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:50.156959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c34223-f0d6-4fe0-a6e8-f4fd7dd5388d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:50.162707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6570cf-6062-47bf-92eb-556a55815e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:50.164069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6570cf-6062-47bf-92eb-556a55815e22 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.165335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6570cf-6062-47bf-92eb-556a55815e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.165782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6570cf-6062-47bf-92eb-556a55815e22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.166827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6570cf-6062-47bf-92eb-556a55815e22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:50.170189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6570cf-6062-47bf-92eb-556a55815e22 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:50.172716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639377c-8227-4c37-8562-635001a4d325 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:50.174883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6639377c-8227-4c37-8562-635001a4d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:50.205054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 898/1795] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:50.206221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6570cf-6062-47bf-92eb-556a55815e22 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:50.219987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6570cf-6062-47bf-92eb-556a55815e22 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:50.235989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba70085-1ab8-48c3-80ff-346de65a1bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:50.236984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba70085-1ab8-48c3-80ff-346de65a1bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.238073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba70085-1ab8-48c3-80ff-346de65a1bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.238432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba70085-1ab8-48c3-80ff-346de65a1bd9 admin admin] PATH_INFO: `/v3/projects/381da679be7a4c7ea65a27a926c88eeb/users/034054c98dee4e0e947063e9726c8322/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.261462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba70085-1ab8-48c3-80ff-346de65a1bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=381da679be7a4c7ea65a27a926c88eeb, user_id=034054c98dee4e0e947063e9726c8322, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:50.265231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba70085-1ab8-48c3-80ff-346de65a1bd9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:50.281928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639377c-8227-4c37-8562-635001a4d325 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:50.297350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 899/1796] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:50 2022] PUT /identity/v3/projects/381da679be7a4c7ea65a27a926c88eeb/users/034054c98dee4e0e947063e9726c8322/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:50.302132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba70085-1ab8-48c3-80ff-346de65a1bd9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:50.328616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:50.389417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e8811-e5fe-4e7a-a35f-feb6acf4f8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:50.391601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e8811-e5fe-4e7a-a35f-feb6acf4f8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.391942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e8811-e5fe-4e7a-a35f-feb6acf4f8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.392229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e8811-e5fe-4e7a-a35f-feb6acf4f8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.460647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba70085-1ab8-48c3-80ff-346de65a1bd9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:50.474686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbb67b-1dc7-4aba-9671-50b96f8492aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:50.476187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb67b-1dc7-4aba-9671-50b96f8492aa admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.477074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb67b-1dc7-4aba-9671-50b96f8492aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.478547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbb67b-1dc7-4aba-9671-50b96f8492aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.482711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbb67b-1dc7-4aba-9671-50b96f8492aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:50.488747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fbb67b-1dc7-4aba-9671-50b96f8492aa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:50.515653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e8811-e5fe-4e7a-a35f-feb6acf4f8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:50.517740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e8811-e5fe-4e7a-a35f-feb6acf4f8d5 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:50.521653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 900/1797] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:50.524487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbb67b-1dc7-4aba-9671-50b96f8492aa admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:50.536631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 898/1798] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:50.538860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbb67b-1dc7-4aba-9671-50b96f8492aa admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:50.546993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:50.548022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.548377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.548726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] PATH_INFO: `/v3/projects/381da679be7a4c7ea65a27a926c88eeb/users/034054c98dee4e0e947063e9726c8322/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.551432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=381da679be7a4c7ea65a27a926c88eeb, user_id=034054c98dee4e0e947063e9726c8322, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:50.554074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:50.569125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 901/1799] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:50 2022] PUT /identity/v3/projects/381da679be7a4c7ea65a27a926c88eeb/users/034054c98dee4e0e947063e9726c8322/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:50.582053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291677e-c2b9-4426-b460-09359ddcd885 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.582341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291677e-c2b9-4426-b460-09359ddcd885 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.582663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7291677e-c2b9-4426-b460-09359ddcd885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:50.628528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7291677e-c2b9-4426-b460-09359ddcd885 None None] MFA Rules not processed for user `034054c98dee4e0e947063e9726c8322`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:50.632509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7291677e-c2b9-4426-b460-09359ddcd885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:50.767283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:50.820868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 899/1800] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:50 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:50.831508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:50.834137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:50.972547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136ec9a-5135-4c99-9771-732de0acb010 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:50.982782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e959b6d-448f-4634-aba3-b2ad7ec340f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:50.983184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959b6d-448f-4634-aba3-b2ad7ec340f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:50.983445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959b6d-448f-4634-aba3-b2ad7ec340f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:50.983834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e959b6d-448f-4634-aba3-b2ad7ec340f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:51.004115 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e959b6d-448f-4634-aba3-b2ad7ec340f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:51.004958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e959b6d-448f-4634-aba3-b2ad7ec340f9 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:51.047712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 902/1801] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:50 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:51.322638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291677e-c2b9-4426-b460-09359ddcd885 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:51.327924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7291677e-c2b9-4426-b460-09359ddcd885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:51.341634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e959b6d-448f-4634-aba3-b2ad7ec340f9 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:51.360599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e959b6d-448f-4634-aba3-b2ad7ec340f9 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:51.376334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df61465-924e-43df-b37c-931c4ff4fb0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:51.376334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61465-924e-43df-b37c-931c4ff4fb0a service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:51.376334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61465-924e-43df-b37c-931c4ff4fb0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:51.376334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df61465-924e-43df-b37c-931c4ff4fb0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:51.376939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9df61465-924e-43df-b37c-931c4ff4fb0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:51.537578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df61465-924e-43df-b37c-931c4ff4fb0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:51.540086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df61465-924e-43df-b37c-931c4ff4fb0a service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:51.552827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 903/1802] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:51 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:22:51.654632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7291677e-c2b9-4426-b460-09359ddcd885 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:51.677067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:51.679350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:51.684914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:51.685657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] PATH_INFO: `/v3/projects/76cb621543b1486e95c0ec193887bf9d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:51.691317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76cb621543b1486e95c0ec193887bf9d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:51.700009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:51.722613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a71f3f-e7d8-44aa-abe6-d303786ce5d5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:51.723142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a71f3f-e7d8-44aa-abe6-d303786ce5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:51.723595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a71f3f-e7d8-44aa-abe6-d303786ce5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:51.767255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-52a71f3f-e7d8-44aa-abe6-d303786ce5d5 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:51.772303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-52a71f3f-e7d8-44aa-abe6-d303786ce5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:51.844891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76cb621543b1486e95c0ec193887bf9d', 'request_id': 'req-813a7496-69f2-495b-be71-b9f2e789ac54'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:51.852591 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:51.861750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76cb621543b1486e95c0ec193887bf9d', 'request_id': 'req-813a7496-69f2-495b-be71-b9f2e789ac54'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:51.882054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76cb621543b1486e95c0ec193887bf9d', 'request_id': 'req-813a7496-69f2-495b-be71-b9f2e789ac54'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:51.906043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] The token cache is being invalidate because project 76cb621543b1486e95c0ec193887bf9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:51.906410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:51.910186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:51.925734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 900/1803] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:51 2022] DELETE /identity/v3/projects/76cb621543b1486e95c0ec193887bf9d => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:51.927909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:51.934466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:52.063659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 904/1804] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:22:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:52.138633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a7496-69f2-495b-be71-b9f2e789ac54 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:52.174360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:52.180272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:52.181287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:52.182191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] PATH_INFO: `/v3/users/d02898c0baba4567ad3a6ac6ad77a0fa` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:52.187718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02898c0baba4567ad3a6ac6ad77a0fa)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:52.191489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:52.225023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:52.354375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02898c0baba4567ad3a6ac6ad77a0fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:52.356622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02898c0baba4567ad3a6ac6ad77a0fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:52.359284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02898c0baba4567ad3a6ac6ad77a0fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:52.405811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02898c0baba4567ad3a6ac6ad77a0fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:52.414045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a71f3f-e7d8-44aa-abe6-d303786ce5d5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:52.418530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-52a71f3f-e7d8-44aa-abe6-d303786ce5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:52.452120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02898c0baba4567ad3a6ac6ad77a0fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:52.567385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 901/1805] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:51 2022] DELETE /identity/v3/users/d02898c0baba4567ad3a6ac6ad77a0fa => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:52.597356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:52.599096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:52.765423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a71f3f-e7d8-44aa-abe6-d303786ce5d5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:52.872293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2aa77d-9c78-47dc-8048-769305127d9f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:52.890973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:52.904388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:52.906979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:52.906979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] PATH_INFO: `/v3/users/c833e44e43d14f7c86ad0d42217c93ce` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:52.913740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c833e44e43d14f7c86ad0d42217c93ce)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:52.918587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:52.979341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897d30e-2a4a-48f7-8730-bfff5808b263 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:52.981030 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897d30e-2a4a-48f7-8730-bfff5808b263 service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:52.982500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897d30e-2a4a-48f7-8730-bfff5808b263 service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:52.982793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897d30e-2a4a-48f7-8730-bfff5808b263 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:52.990929 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5897d30e-2a4a-48f7-8730-bfff5808b263 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:53.095365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c833e44e43d14f7c86ad0d42217c93ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:53.096614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c833e44e43d14f7c86ad0d42217c93ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:53.104176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c833e44e43d14f7c86ad0d42217c93ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:53.148080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c833e44e43d14f7c86ad0d42217c93ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:53.160443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897d30e-2a4a-48f7-8730-bfff5808b263 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:53.164068 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5897d30e-2a4a-48f7-8730-bfff5808b263 service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:53.172146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c833e44e43d14f7c86ad0d42217c93ce'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:53.191043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 905/1806] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:22:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 777 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:22:53.256718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 902/1807] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:52 2022] DELETE /identity/v3/users/c833e44e43d14f7c86ad0d42217c93ce => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:22:53.261738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461dd94-91d4-46e9-93af-44afc33a4e42 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:53.262029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461dd94-91d4-46e9-93af-44afc33a4e42 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:53.262309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461dd94-91d4-46e9-93af-44afc33a4e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:53.315299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7461dd94-91d4-46e9-93af-44afc33a4e42 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:53.319484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7461dd94-91d4-46e9-93af-44afc33a4e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:53.377501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:53.624087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 906/1808] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:54.027645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:54.029244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:54.060517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:54.314398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d2dc78-5318-4e3d-a048-e0a68c083229 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:54.320932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:54.330267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:54.333108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:54.334004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:54.362525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:54.372985 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:54.453518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 903/1809] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:54.595024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6b1f-9610-4c64-9142-9faaaf46a6ff None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:54.603033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6b1f-9610-4c64-9142-9faaaf46a6ff None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:54.603033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6b1f-9610-4c64-9142-9faaaf46a6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:54.685404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-879a6b1f-9610-4c64-9142-9faaaf46a6ff None None] MFA Rules not processed for user `72fb6996e1974060b162070f8da365aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:54.686887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-879a6b1f-9610-4c64-9142-9faaaf46a6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:54.729586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 907/1810] 10.222.0.81 () {58 vars in 1015 bytes} [Thu Jun 16 17:22:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:55.490518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:55.492443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:55.519734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcbbf6-42f3-4757-84d9-1f7e7415c548 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:55.543538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:55.545227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:55.546219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:55.546775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] PATH_INFO: `/v3/projects/f3afc9b4f63a4595af85b9746cf1d24f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:55.548070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3afc9b4f63a4595af85b9746cf1d24f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:55.551711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:55.607337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3afc9b4f63a4595af85b9746cf1d24f', 'request_id': 'req-a9e5cf06-613a-4701-8ce0-ad36c051b93c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:55.615663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3afc9b4f63a4595af85b9746cf1d24f', 'request_id': 'req-a9e5cf06-613a-4701-8ce0-ad36c051b93c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:55.627430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3afc9b4f63a4595af85b9746cf1d24f', 'request_id': 'req-a9e5cf06-613a-4701-8ce0-ad36c051b93c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:55.649429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] The token cache is being invalidate because project f3afc9b4f63a4595af85b9746cf1d24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:55.649926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:55.650758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:55.653169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 904/1811] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:55 2022] DELETE /identity/v3/projects/f3afc9b4f63a4595af85b9746cf1d24f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:55.658750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9676a-23c2-4eeb-8d9f-fb1045289a42 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:55.659287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9676a-23c2-4eeb-8d9f-fb1045289a42 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:55.659977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9676a-23c2-4eeb-8d9f-fb1045289a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:55.686196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-86a9676a-23c2-4eeb-8d9f-fb1045289a42 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:55.688310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-86a9676a-23c2-4eeb-8d9f-fb1045289a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:55.798995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:55.800776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:55.852910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 908/1812] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:55.861872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df978-1414-4d6e-bfcf-79e997cc949c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:55.862385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df978-1414-4d6e-bfcf-79e997cc949c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:55.864117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4df978-1414-4d6e-bfcf-79e997cc949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:55.888536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-fb4df978-1414-4d6e-bfcf-79e997cc949c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:55.892480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fb4df978-1414-4d6e-bfcf-79e997cc949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:55.913242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 909/1813] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:22:55.916724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df978-1414-4d6e-bfcf-79e997cc949c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:55.933140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e5cf06-613a-4701-8ce0-ad36c051b93c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:55.938755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:55.939815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:55.940238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:55.940340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4df978-1414-4d6e-bfcf-79e997cc949c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:55.940670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] PATH_INFO: `/v3/projects/15b95a6aa1794c16afee59c287754fd2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:55.942288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15b95a6aa1794c16afee59c287754fd2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:55.944182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:55.974098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:55.974791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:55.975073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:55.975349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:55.976141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:55.976397 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:55.978737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:55.994773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 910/1814] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:56.007704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:56.011102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:56.029847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b54667-74b4-43c5-b6bd-3fab88820da0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:56.040596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15b95a6aa1794c16afee59c287754fd2', 'request_id': 'req-53a15415-d188-4c86-9106-e4939edc9291'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.058510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15b95a6aa1794c16afee59c287754fd2', 'request_id': 'req-53a15415-d188-4c86-9106-e4939edc9291'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.070909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15b95a6aa1794c16afee59c287754fd2', 'request_id': 'req-53a15415-d188-4c86-9106-e4939edc9291'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.090883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] The token cache is being invalidate because project 15b95a6aa1794c16afee59c287754fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:56.091182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.091868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.094030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 905/1815] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:55 2022] DELETE /identity/v3/projects/15b95a6aa1794c16afee59c287754fd2 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:56.098113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:56.102876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:56.159902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:56.160956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.161437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.162241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.163118 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:56.170905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:56.226588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 911/1816] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:56 2022] POST /identity/v3/projects => generated 377 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:56.229398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:56.231351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:56.236174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a15415-d188-4c86-9106-e4939edc9291 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:56.255694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:56.327172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:56.328762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.329284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.329749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] PATH_INFO: `/v3/users/7c2d66447744453fbe6a5dd8ed88fb9a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.331903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2d66447744453fbe6a5dd8ed88fb9a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:56.336410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:56.362235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f5b55-3de8-41ef-b2e8-14aa0a2a86ac admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:56.386256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:56.389892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.390382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.392160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.395307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:56.407631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:56.438528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2d66447744453fbe6a5dd8ed88fb9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.439494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2d66447744453fbe6a5dd8ed88fb9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.440467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2d66447744453fbe6a5dd8ed88fb9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.458656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 912/1817] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:56 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:22:56.463616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:56.467266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:56.482191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2d66447744453fbe6a5dd8ed88fb9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.504944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2d66447744453fbe6a5dd8ed88fb9a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:56.507998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefb14f1-2369-4da9-820d-716ff241bff8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:56.509139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:56.522333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:56.523037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.524424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.526304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.527663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:56.530552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:56.543963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 906/1818] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:56 2022] DELETE /identity/v3/users/7c2d66447744453fbe6a5dd8ed88fb9a => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:22:56.555977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 913/1819] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:56.562303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:56.725105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6e3e1-9cbf-4f36-844e-f5870d5cefcf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:56.732460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:56.738140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36b888-fa02-4285-8ee0-a58dc82d2fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:56.739482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36b888-fa02-4285-8ee0-a58dc82d2fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.739823 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36b888-fa02-4285-8ee0-a58dc82d2fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.740202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36b888-fa02-4285-8ee0-a58dc82d2fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.741977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36b888-fa02-4285-8ee0-a58dc82d2fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:56.743041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:56.744841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b36b888-fa02-4285-8ee0-a58dc82d2fb3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:56.786794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 907/1820] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:56.787840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36b888-fa02-4285-8ee0-a58dc82d2fb3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:56.801641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36b888-fa02-4285-8ee0-a58dc82d2fb3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:56.809683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bfb30d-1740-4e0e-b0f6-4a8c73dfaee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:56.810775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfb30d-1740-4e0e-b0f6-4a8c73dfaee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.811205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfb30d-1740-4e0e-b0f6-4a8c73dfaee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.812143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bfb30d-1740-4e0e-b0f6-4a8c73dfaee5 admin admin] PATH_INFO: `/v3/projects/7b0cc6ed0d284d9e9c8a2d56a556ed04/users/ce5009b738ff4cff8b0d143ea0049756/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.845615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfb30d-1740-4e0e-b0f6-4a8c73dfaee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b0cc6ed0d284d9e9c8a2d56a556ed04, user_id=ce5009b738ff4cff8b0d143ea0049756, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:56.852989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bfb30d-1740-4e0e-b0f6-4a8c73dfaee5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:56.884868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 908/1821] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:56 2022] PUT /identity/v3/projects/7b0cc6ed0d284d9e9c8a2d56a556ed04/users/ce5009b738ff4cff8b0d143ea0049756/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:56.897915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0c1ca-5834-45c7-ae5a-70ad4acfca41 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:56.898490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0c1ca-5834-45c7-ae5a-70ad4acfca41 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:56.899469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa0c1ca-5834-45c7-ae5a-70ad4acfca41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:56.921077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782cbcd-ab33-4ed3-829e-ae12b73c7d1e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:56.958868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1fa0c1ca-5834-45c7-ae5a-70ad4acfca41 None None] MFA Rules not processed for user `ce5009b738ff4cff8b0d143ea0049756`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:56.960887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1fa0c1ca-5834-45c7-ae5a-70ad4acfca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:57.030726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.031554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.031842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.032299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.035017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:57.058593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.058769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.068980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 914/1822] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:22:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:22:57.091568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 909/1823] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:57.092753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.156665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0c1ca-5834-45c7-ae5a-70ad4acfca41 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.158914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1fa0c1ca-5834-45c7-ae5a-70ad4acfca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:57.199876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:57.208620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03fb64-4c07-4264-90c3-58c84fd44c90 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.216767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176590c-b279-4d98-a2c5-734845c471e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.217603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176590c-b279-4d98-a2c5-734845c471e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.217884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176590c-b279-4d98-a2c5-734845c471e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.218161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176590c-b279-4d98-a2c5-734845c471e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.219009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176590c-b279-4d98-a2c5-734845c471e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.221333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176590c-b279-4d98-a2c5-734845c471e4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.234737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:57.258948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa0c1ca-5834-45c7-ae5a-70ad4acfca41 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.259926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 915/1824] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:57 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:57.278349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176590c-b279-4d98-a2c5-734845c471e4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.415401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.416783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.417348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.417892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] PATH_INFO: `/v3/projects/c69880e6ee844f8da325ab4902350dc2` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.420988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c69880e6ee844f8da325ab4902350dc2)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.429477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.484598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176590c-b279-4d98-a2c5-734845c471e4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.501665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108bd64-2bbf-483e-aa5a-a86c58e7d18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.502363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108bd64-2bbf-483e-aa5a-a86c58e7d18b admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.502645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108bd64-2bbf-483e-aa5a-a86c58e7d18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.502915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108bd64-2bbf-483e-aa5a-a86c58e7d18b admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.503616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108bd64-2bbf-483e-aa5a-a86c58e7d18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.505589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108bd64-2bbf-483e-aa5a-a86c58e7d18b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.548626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 916/1825] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:57 2022] POST /identity/v3/users => generated 290 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:57.552642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108bd64-2bbf-483e-aa5a-a86c58e7d18b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.565305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108bd64-2bbf-483e-aa5a-a86c58e7d18b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.577436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c69880e6ee844f8da325ab4902350dc2', 'request_id': 'req-f9d52d92-eefc-4662-8948-34690f90c20b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:57.588677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c69880e6ee844f8da325ab4902350dc2', 'request_id': 'req-f9d52d92-eefc-4662-8948-34690f90c20b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:57.600198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c69880e6ee844f8da325ab4902350dc2', 'request_id': 'req-f9d52d92-eefc-4662-8948-34690f90c20b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:57.623916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] The token cache is being invalidate because project c69880e6ee844f8da325ab4902350dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:22:57.624260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:57.625057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f9d52d92-eefc-4662-8948-34690f90c20b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:57.629769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 910/1826] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:22:57 2022] DELETE /identity/v3/projects/c69880e6ee844f8da325ab4902350dc2 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:57.635962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680f1f-8515-4e39-a37c-f5dc7f3cf1ce None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.636758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680f1f-8515-4e39-a37c-f5dc7f3cf1ce None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.637915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5680f1f-8515-4e39-a37c-f5dc7f3cf1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.687946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f5680f1f-8515-4e39-a37c-f5dc7f3cf1ce None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:57.699047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.699047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.699356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5680f1f-8515-4e39-a37c-f5dc7f3cf1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:57.699451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.699451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.699451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.699451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.736129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 911/1827] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:57.736348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 917/1828] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:57.739319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.740021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2b863-c51c-4f8d-a318-cabd203fc96d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.740408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2b863-c51c-4f8d-a318-cabd203fc96d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.740595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2b863-c51c-4f8d-a318-cabd203fc96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.757238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:57.783333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4df2b863-c51c-4f8d-a318-cabd203fc96d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:57.787014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4df2b863-c51c-4f8d-a318-cabd203fc96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:57.807954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf02fb-151c-4d0c-98e5-e73a8206e316 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.811195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 912/1829] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:57.813992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2b863-c51c-4f8d-a318-cabd203fc96d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.822259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0322d2-977d-4e7b-b811-68705b7fc025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.822993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0322d2-977d-4e7b-b811-68705b7fc025 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.823323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0322d2-977d-4e7b-b811-68705b7fc025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.823605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0322d2-977d-4e7b-b811-68705b7fc025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.824358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0322d2-977d-4e7b-b811-68705b7fc025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.826344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0322d2-977d-4e7b-b811-68705b7fc025 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.841331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2b863-c51c-4f8d-a318-cabd203fc96d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.847887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.848802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.849087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.849404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.850195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:22:57.850507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.850616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 918/1830] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:57.852471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.856217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0322d2-977d-4e7b-b811-68705b7fc025 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.885933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 913/1831] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:22:57.891712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:57.892566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0322d2-977d-4e7b-b811-68705b7fc025 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.903665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed0fa5-35bf-4cfd-aca2-b3e68d0bb6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.904177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6958d1f-9e9d-48e3-89f5-303e42d3cce6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:57.907623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed0fa5-35bf-4cfd-aca2-b3e68d0bb6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.909684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120341d-e57c-4376-a444-4f7159ba3fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:57.910484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120341d-e57c-4376-a444-4f7159ba3fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:57.910840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120341d-e57c-4376-a444-4f7159ba3fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.911191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3120341d-e57c-4376-a444-4f7159ba3fcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.911979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120341d-e57c-4376-a444-4f7159ba3fcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:57.914002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3120341d-e57c-4376-a444-4f7159ba3fcf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:57.920688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed0fa5-35bf-4cfd-aca2-b3e68d0bb6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:57.921450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed0fa5-35bf-4cfd-aca2-b3e68d0bb6fd admin admin] PATH_INFO: `/v3/projects/2da941bdf37c4adab2720eda49c6f8bd/users/8275dbd39af2404482029bf180bf3796/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:57.991556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed0fa5-35bf-4cfd-aca2-b3e68d0bb6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da941bdf37c4adab2720eda49c6f8bd, user_id=8275dbd39af2404482029bf180bf3796, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:58.001982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed0fa5-35bf-4cfd-aca2-b3e68d0bb6fd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:58.007237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 914/1832] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:57 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:58.009792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120341d-e57c-4376-a444-4f7159ba3fcf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:58.054323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 919/1833] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:57 2022] PUT /identity/v3/projects/2da941bdf37c4adab2720eda49c6f8bd/users/8275dbd39af2404482029bf180bf3796/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:58.067746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8794-ebf4-4032-8135-1c109ff3593a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:58.068279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8794-ebf4-4032-8135-1c109ff3593a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:58.068705 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef8794-ebf4-4032-8135-1c109ff3593a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:58.198202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-50ef8794-ebf4-4032-8135-1c109ff3593a None None] MFA Rules not processed for user `8275dbd39af2404482029bf180bf3796`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:58.202688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-50ef8794-ebf4-4032-8135-1c109ff3593a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:58.252742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:58.310156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:58.340181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3120341d-e57c-4376-a444-4f7159ba3fcf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:58.569653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 920/1834] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:22:58.570802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef8794-ebf4-4032-8135-1c109ff3593a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:58.589789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dea44-1508-428e-a5d7-aa9f222f6f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:58.592078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dea44-1508-428e-a5d7-aa9f222f6f06 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:58.596974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dea44-1508-428e-a5d7-aa9f222f6f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:58.597554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66dea44-1508-428e-a5d7-aa9f222f6f06 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:58.602216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66dea44-1508-428e-a5d7-aa9f222f6f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:58.604392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66dea44-1508-428e-a5d7-aa9f222f6f06 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:58.627859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef8794-ebf4-4032-8135-1c109ff3593a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:58.645330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960e14e-a0b4-4d10-8fa6-7c964634a1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:58.646099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960e14e-a0b4-4d10-8fa6-7c964634a1e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:58.649859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960e14e-a0b4-4d10-8fa6-7c964634a1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:58.650235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960e14e-a0b4-4d10-8fa6-7c964634a1e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:58.651145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960e14e-a0b4-4d10-8fa6-7c964634a1e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:58.666269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e960e14e-a0b4-4d10-8fa6-7c964634a1e5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:58.669378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 915/1835] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:58 2022] POST /identity/v3/users => generated 293 bytes in 661 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:58.673246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dea44-1508-428e-a5d7-aa9f222f6f06 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:58.705741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66dea44-1508-428e-a5d7-aa9f222f6f06 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:58.720330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:58.722445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:58.722751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:58.723035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:58.725629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:58.726862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:58.750224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 916/1836] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:58.751180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:58.762528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:58.789123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 921/1837] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:58 2022] POST /identity/v3/projects => generated 379 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:58.790950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960e14e-a0b4-4d10-8fa6-7c964634a1e5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:58.795823 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae89fc4-53ff-4460-8159-f1f7a7fa4b1e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:58.963133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960e14e-a0b4-4d10-8fa6-7c964634a1e5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:58.972475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:58.974144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:58.975515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:58.975925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39bac6-ccba-4ed5-8914-ab0a33795c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:58.976244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] PATH_INFO: `/v3/users/e4f9d0114da342c8b71842da45834f26` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:58.976787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39bac6-ccba-4ed5-8914-ab0a33795c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:58.977855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39bac6-ccba-4ed5-8914-ab0a33795c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:58.978228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39bac6-ccba-4ed5-8914-ab0a33795c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:58.979127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39bac6-ccba-4ed5-8914-ab0a33795c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:58.979561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f9d0114da342c8b71842da45834f26)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:58.981449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39bac6-ccba-4ed5-8914-ab0a33795c69 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:58.985809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:59.023829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 922/1838] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:59.029774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39bac6-ccba-4ed5-8914-ab0a33795c69 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:59.082507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39bac6-ccba-4ed5-8914-ab0a33795c69 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:59.093756 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8892f2-61ee-4681-a18f-f563a763c2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:59.095085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8892f2-61ee-4681-a18f-f563a763c2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.095570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8892f2-61ee-4681-a18f-f563a763c2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.096032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8892f2-61ee-4681-a18f-f563a763c2f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.097872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8892f2-61ee-4681-a18f-f563a763c2f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:59.100018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8892f2-61ee-4681-a18f-f563a763c2f7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:59.136144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 923/1839] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:59 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:59.138045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8892f2-61ee-4681-a18f-f563a763c2f7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:59.152485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8892f2-61ee-4681-a18f-f563a763c2f7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:59.159136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:59.160451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.160937 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.161470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] PATH_INFO: `/v3/projects/3146fbcf250742c7b9232da34a6a94de/users/75ef5eae667f4e9e986f191fb5beec36/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.181783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3146fbcf250742c7b9232da34a6a94de, user_id=75ef5eae667f4e9e986f191fb5beec36, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:59.185019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:59.201693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f9d0114da342c8b71842da45834f26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:59.202640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f9d0114da342c8b71842da45834f26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:59.205616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f9d0114da342c8b71842da45834f26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:59.216747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 924/1840] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:59 2022] PUT /identity/v3/projects/3146fbcf250742c7b9232da34a6a94de/users/75ef5eae667f4e9e986f191fb5beec36/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:59.221944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:59.225586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:59.249304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f9d0114da342c8b71842da45834f26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:59.262591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:59.275223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cdf95dff-9d37-4bdb-8c3c-359b2526911a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f9d0114da342c8b71842da45834f26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:22:59.306393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 917/1841] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:22:58 2022] DELETE /identity/v3/users/e4f9d0114da342c8b71842da45834f26 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:59.307970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb818f0-2f24-412c-98fa-a22a60e502d8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.308261 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb818f0-2f24-412c-98fa-a22a60e502d8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.308562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb818f0-2f24-412c-98fa-a22a60e502d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.351571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:22:59.369836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7cb818f0-2f24-412c-98fa-a22a60e502d8 None None] MFA Rules not processed for user `75ef5eae667f4e9e986f191fb5beec36`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:59.371092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7cb818f0-2f24-412c-98fa-a22a60e502d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:59.371704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fb6b72-0a48-4cd1-9382-35024b0f9a01 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:59.624246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349cf0b-a279-40e3-9928-719f08aac03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:59.626293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349cf0b-a279-40e3-9928-719f08aac03a admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.628104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349cf0b-a279-40e3-9928-719f08aac03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.629824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349cf0b-a279-40e3-9928-719f08aac03a admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.635173 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349cf0b-a279-40e3-9928-719f08aac03a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:59.636879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2349cf0b-a279-40e3-9928-719f08aac03a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:59.677150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 925/1842] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:22:59.681844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803d2d4-169c-40bf-b53e-be35406bfb63 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.682125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803d2d4-169c-40bf-b53e-be35406bfb63 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.687252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 918/1843] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:59.688031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb818f0-2f24-412c-98fa-a22a60e502d8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:59.688893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803d2d4-169c-40bf-b53e-be35406bfb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.719643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb818f0-2f24-412c-98fa-a22a60e502d8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:59.733380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672767a4-2b1a-4253-b2b5-568633b5d0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:59.735331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672767a4-2b1a-4253-b2b5-568633b5d0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.736419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672767a4-2b1a-4253-b2b5-568633b5d0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.737787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672767a4-2b1a-4253-b2b5-568633b5d0d7 admin admin] PATH_INFO: `/v3/projects/927236d7cbc048f68e805499cafc0f52/users/51ebf734061848d6834b9ee5d3409b9d/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.739698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6803d2d4-169c-40bf-b53e-be35406bfb63 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:22:59.744396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6803d2d4-169c-40bf-b53e-be35406bfb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:59.770372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 926/1844] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:22:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:22:59.771079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803d2d4-169c-40bf-b53e-be35406bfb63 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:59.775640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6803d2d4-169c-40bf-b53e-be35406bfb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:22:59.793473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672767a4-2b1a-4253-b2b5-568633b5d0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927236d7cbc048f68e805499cafc0f52, user_id=51ebf734061848d6834b9ee5d3409b9d, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:59.802489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672767a4-2b1a-4253-b2b5-568633b5d0d7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:59.827316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803d2d4-169c-40bf-b53e-be35406bfb63 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:22:59.842701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f2f1d-c5f2-42e4-9315-66eed8c736a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:22:59.844570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 919/1845] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:22:59 2022] PUT /identity/v3/projects/927236d7cbc048f68e805499cafc0f52/users/51ebf734061848d6834b9ee5d3409b9d/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:22:59.845160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f2f1d-c5f2-42e4-9315-66eed8c736a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:22:59.845482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f2f1d-c5f2-42e4-9315-66eed8c736a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:22:59.845766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01f2f1d-c5f2-42e4-9315-66eed8c736a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:22:59.846863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f2f1d-c5f2-42e4-9315-66eed8c736a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:22:59.848558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672767a4-2b1a-4253-b2b5-568633b5d0d7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:22:59.854473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01f2f1d-c5f2-42e4-9315-66eed8c736a3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:22:59.918768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 927/1846] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:22:59 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:22:59.924766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f2f1d-c5f2-42e4-9315-66eed8c736a3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.023284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672767a4-2b1a-4253-b2b5-568633b5d0d7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.096532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01f2f1d-c5f2-42e4-9315-66eed8c736a3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.108531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.110589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.111268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.112277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.115618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.120473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.173399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:00.175423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 928/1847] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:22:59 2022] POST /identity/v3/users => generated 299 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:00.177983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.187761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:00.197663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93533f7d-64ba-43b1-9d6a-8aa9573d12f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.198513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93533f7d-64ba-43b1-9d6a-8aa9573d12f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.200071 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93533f7d-64ba-43b1-9d6a-8aa9573d12f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.200366 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93533f7d-64ba-43b1-9d6a-8aa9573d12f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.203121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93533f7d-64ba-43b1-9d6a-8aa9573d12f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.205936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93533f7d-64ba-43b1-9d6a-8aa9573d12f9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.245781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 920/1848] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:00.247773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93533f7d-64ba-43b1-9d6a-8aa9573d12f9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.280928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93533f7d-64ba-43b1-9d6a-8aa9573d12f9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.295674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf342396-1dd9-46b7-8967-4bd0a6f8a649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.296799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342396-1dd9-46b7-8967-4bd0a6f8a649 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.297099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342396-1dd9-46b7-8967-4bd0a6f8a649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.297875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf342396-1dd9-46b7-8967-4bd0a6f8a649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.300852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342396-1dd9-46b7-8967-4bd0a6f8a649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.302643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf342396-1dd9-46b7-8967-4bd0a6f8a649 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.337251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 921/1849] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:00.338459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf342396-1dd9-46b7-8967-4bd0a6f8a649 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.361135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf342396-1dd9-46b7-8967-4bd0a6f8a649 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.363461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8845f-cde1-4855-af87-323d3441985f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.366930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5aaaf-7ffa-4b2b-ae6b-8ed53e9a2ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.369200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5aaaf-7ffa-4b2b-ae6b-8ed53e9a2ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.369864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5aaaf-7ffa-4b2b-ae6b-8ed53e9a2ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.371148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5aaaf-7ffa-4b2b-ae6b-8ed53e9a2ef1 admin admin] PATH_INFO: `/v3/projects/927236d7cbc048f68e805499cafc0f52/users/51ebf734061848d6834b9ee5d3409b9d/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.373940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5aaaf-7ffa-4b2b-ae6b-8ed53e9a2ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927236d7cbc048f68e805499cafc0f52, user_id=51ebf734061848d6834b9ee5d3409b9d, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.377274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5aaaf-7ffa-4b2b-ae6b-8ed53e9a2ef1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.386440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157f850-ee5c-4edd-aefa-edad906c69c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.387277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f850-ee5c-4edd-aefa-edad906c69c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.396468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f850-ee5c-4edd-aefa-edad906c69c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.396891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 922/1850] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:00 2022] PUT /identity/v3/projects/927236d7cbc048f68e805499cafc0f52/users/51ebf734061848d6834b9ee5d3409b9d/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:00.397479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f850-ee5c-4edd-aefa-edad906c69c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.398090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5aaaf-7ffa-4b2b-ae6b-8ed53e9a2ef1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.419953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157f850-ee5c-4edd-aefa-edad906c69c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.421453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157f850-ee5c-4edd-aefa-edad906c69c3 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.460072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 929/1851] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:00.476748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4c205-a3a9-4be4-8f95-8eaa34187b7d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.476748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4c205-a3a9-4be4-8f95-8eaa34187b7d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.476748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4c205-a3a9-4be4-8f95-8eaa34187b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.487178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:00.539951 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a4d4c205-a3a9-4be4-8f95-8eaa34187b7d None None] MFA Rules not processed for user `51ebf734061848d6834b9ee5d3409b9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:00.540402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a4d4c205-a3a9-4be4-8f95-8eaa34187b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:00.542716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5aaaf-7ffa-4b2b-ae6b-8ed53e9a2ef1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.553841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3251e62-5b2e-4218-a1be-c488f79c578d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.555951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3251e62-5b2e-4218-a1be-c488f79c578d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.556236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3251e62-5b2e-4218-a1be-c488f79c578d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.556525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3251e62-5b2e-4218-a1be-c488f79c578d admin admin] PATH_INFO: `/v3/projects/4fafcf8cd87846dcb8972c7861146706/users/ea71cda5df934fc791c3fa6be9cac60c/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.615291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3251e62-5b2e-4218-a1be-c488f79c578d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fafcf8cd87846dcb8972c7861146706, user_id=ea71cda5df934fc791c3fa6be9cac60c, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.618449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3251e62-5b2e-4218-a1be-c488f79c578d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.650574 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 923/1852] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:00 2022] PUT /identity/v3/projects/4fafcf8cd87846dcb8972c7861146706/users/ea71cda5df934fc791c3fa6be9cac60c/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:00.658554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3251e62-5b2e-4218-a1be-c488f79c578d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.690866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 930/1853] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:00.701475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4c205-a3a9-4be4-8f95-8eaa34187b7d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.705395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a4d4c205-a3a9-4be4-8f95-8eaa34187b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:00.850568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3251e62-5b2e-4218-a1be-c488f79c578d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.878348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f5638-1592-4b73-ad84-810021512434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.882201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5638-1592-4b73-ad84-810021512434 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.883243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5638-1592-4b73-ad84-810021512434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.883629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5638-1592-4b73-ad84-810021512434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.887658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f5638-1592-4b73-ad84-810021512434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.891265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f5638-1592-4b73-ad84-810021512434 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:00.915207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4c205-a3a9-4be4-8f95-8eaa34187b7d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.928811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 924/1854] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:00.930081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f5638-1592-4b73-ad84-810021512434 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:00.968624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f5638-1592-4b73-ad84-810021512434 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:00.983695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212d8fc-8cc3-4243-98d2-91d2b1d723ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:00.985368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212d8fc-8cc3-4243-98d2-91d2b1d723ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:00.986249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212d8fc-8cc3-4243-98d2-91d2b1d723ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:00.986869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212d8fc-8cc3-4243-98d2-91d2b1d723ea admin admin] PATH_INFO: `/v3/projects/4fafcf8cd87846dcb8972c7861146706/users/ea71cda5df934fc791c3fa6be9cac60c/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:00.989232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212d8fc-8cc3-4243-98d2-91d2b1d723ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fafcf8cd87846dcb8972c7861146706, user_id=ea71cda5df934fc791c3fa6be9cac60c, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:00.993003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212d8fc-8cc3-4243-98d2-91d2b1d723ea admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:01.016048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 925/1855] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:00 2022] PUT /identity/v3/projects/4fafcf8cd87846dcb8972c7861146706/users/ea71cda5df934fc791c3fa6be9cac60c/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:01.038993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d8ad8-7bd9-4ad0-a5f9-b0c7dd89d420 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:01.039828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d8ad8-7bd9-4ad0-a5f9-b0c7dd89d420 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:01.040417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d8ad8-7bd9-4ad0-a5f9-b0c7dd89d420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:01.055768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:01.056458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:01.056748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:01.057062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:01.106194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8d6d8ad8-7bd9-4ad0-a5f9-b0c7dd89d420 None None] MFA Rules not processed for user `ea71cda5df934fc791c3fa6be9cac60c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:01.107434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8d6d8ad8-7bd9-4ad0-a5f9-b0c7dd89d420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:01.185963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:01.373962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 926/1856] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:01.375695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd28ab-f7bf-45cc-bedf-e870c83da901 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:01.376198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd28ab-f7bf-45cc-bedf-e870c83da901 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:01.376713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd28ab-f7bf-45cc-bedf-e870c83da901 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:01.381795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 927/1857] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:23:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:01.673637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853b076-dda5-467c-ba5a-af6072545231 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:01.673918 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853b076-dda5-467c-ba5a-af6072545231 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:01.674192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853b076-dda5-467c-ba5a-af6072545231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:01.741710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5853b076-dda5-467c-ba5a-af6072545231 None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:01.742920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5853b076-dda5-467c-ba5a-af6072545231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:01.794158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:01.799720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:01.803481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:01.829279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 931/1858] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:00 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 1127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:01.833231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:01.836177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:01.891850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494429c-7b38-4fba-a451-56621ad1282e service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:01.999056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 928/1859] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:23:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:02.000723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853b076-dda5-467c-ba5a-af6072545231 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:02.020417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5853b076-dda5-467c-ba5a-af6072545231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:02.034310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:02.035921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:02.036573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:02.036988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] PATH_INFO: `/v3/users/3333fc6138a746fda98c85441e876ccf` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:02.038571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3333fc6138a746fda98c85441e876ccf)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:02.041689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:02.122677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3333fc6138a746fda98c85441e876ccf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:02.147656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3333fc6138a746fda98c85441e876ccf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:02.148607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3333fc6138a746fda98c85441e876ccf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:02.161781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3333fc6138a746fda98c85441e876ccf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:02.181697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3333fc6138a746fda98c85441e876ccf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:02.200752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 932/1860] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:01 2022] DELETE /identity/v3/users/3333fc6138a746fda98c85441e876ccf => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:02.200752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:02.206362 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:02.281056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853b076-dda5-467c-ba5a-af6072545231 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:02.328186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d85a1-63da-47a8-baec-2502b46d394f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:02.341115 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7a163-eb51-4529-978f-0654616f2b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:02.342163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7a163-eb51-4529-978f-0654616f2b4e service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:02.345831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7a163-eb51-4529-978f-0654616f2b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:02.346385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7a163-eb51-4529-978f-0654616f2b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:02.357126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8af7a163-eb51-4529-978f-0654616f2b4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:02.511940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:02.551875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc951866-241c-4a4c-9bb9-8edacaf58d7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:02.552792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951866-241c-4a4c-9bb9-8edacaf58d7a service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:02.553081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951866-241c-4a4c-9bb9-8edacaf58d7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:02.558961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc951866-241c-4a4c-9bb9-8edacaf58d7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:02.574142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fc951866-241c-4a4c-9bb9-8edacaf58d7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:02.661131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:02.673188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7a163-eb51-4529-978f-0654616f2b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:02.685205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7a163-eb51-4529-978f-0654616f2b4e service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:02.714964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 933/1861] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:02.718395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b64ccf-55cd-452b-9ef9-53611b772be2 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:02.719926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b64ccf-55cd-452b-9ef9-53611b772be2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:02.720900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b64ccf-55cd-452b-9ef9-53611b772be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:02.809101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d3b64ccf-55cd-452b-9ef9-53611b772be2 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:02.813090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d3b64ccf-55cd-452b-9ef9-53611b772be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:02.830483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc951866-241c-4a4c-9bb9-8edacaf58d7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:02.832233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc951866-241c-4a4c-9bb9-8edacaf58d7a service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:02.864675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 929/1862] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 865 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:02.890373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc18673-709b-4985-b4a1-e2a683893cd2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:02.891341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc18673-709b-4985-b4a1-e2a683893cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:02.893278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc18673-709b-4985-b4a1-e2a683893cd2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:02.900437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 930/1863] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:23:02 2022] GET /identity => generated 270 bytes in 35 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:02.903567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc18673-709b-4985-b4a1-e2a683893cd2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:02.911769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-edc18673-709b-4985-b4a1-e2a683893cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:03.054472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 934/1864] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:03.081878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede874c-4b84-459c-bb9a-d1e58e07c3ce None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.082979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede874c-4b84-459c-bb9a-d1e58e07c3ce None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.083327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede874c-4b84-459c-bb9a-d1e58e07c3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.118568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8ede874c-4b84-459c-bb9a-d1e58e07c3ce None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:03.120704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8ede874c-4b84-459c-bb9a-d1e58e07c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:03.218759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc18673-709b-4985-b4a1-e2a683893cd2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:03.239331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:03.240023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.240307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.240632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.243983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:03.268900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:03.270638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:03.286199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 931/1865] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:03.287420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:03.289426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:03.310484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 935/1866] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:23:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:03.311650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede874c-4b84-459c-bb9a-d1e58e07c3ce None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:03.315207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8ede874c-4b84-459c-bb9a-d1e58e07c3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:03.341427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7256f49-3570-4dba-9f52-d143c9305edd service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:03.359547 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:03.363060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.363570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.363983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] PATH_INFO: `/v3/projects/2fcce555e476489288a632478b45bddd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.368896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fcce555e476489288a632478b45bddd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:03.376661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:03.466617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede874c-4b84-459c-bb9a-d1e58e07c3ce None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:03.496729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39b80b-2507-4412-8df2-40416cd9b7d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:03.497734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b80b-2507-4412-8df2-40416cd9b7d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.499085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b80b-2507-4412-8df2-40416cd9b7d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.499621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b80b-2507-4412-8df2-40416cd9b7d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.535654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39b80b-2507-4412-8df2-40416cd9b7d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:03.544136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39b80b-2507-4412-8df2-40416cd9b7d9 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:03.576924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fcce555e476489288a632478b45bddd', 'request_id': 'req-2c61e85c-fd28-4b95-9f3a-927420aa7db3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:03.588165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 936/1867] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:03.592564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fcce555e476489288a632478b45bddd', 'request_id': 'req-2c61e85c-fd28-4b95-9f3a-927420aa7db3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:03.620624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fcce555e476489288a632478b45bddd', 'request_id': 'req-2c61e85c-fd28-4b95-9f3a-927420aa7db3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:03.654972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] The token cache is being invalidate because project 2fcce555e476489288a632478b45bddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:03.655646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:03.656530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:03.658880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 932/1868] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:03 2022] DELETE /identity/v3/projects/2fcce555e476489288a632478b45bddd => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:03.662982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39b80b-2507-4412-8df2-40416cd9b7d9 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:03.668447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:03.674282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39b80b-2507-4412-8df2-40416cd9b7d9 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:03.849997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:03.851222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:03.851809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:03.852400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] PATH_INFO: `/v3/users/a5d45f4527d4486d902c400ee6895d51` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:03.854984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d45f4527d4486d902c400ee6895d51)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:03.861037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:03.972241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d45f4527d4486d902c400ee6895d51'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.006945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d45f4527d4486d902c400ee6895d51'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.010093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d45f4527d4486d902c400ee6895d51'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.024111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d45f4527d4486d902c400ee6895d51'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.044764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d78d1a39-d3e6-45fe-80f4-d854e834ba3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d45f4527d4486d902c400ee6895d51'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.058568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 937/1869] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:03 2022] DELETE /identity/v3/users/a5d45f4527d4486d902c400ee6895d51 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:04.203358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:04.207993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:04.268522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61e85c-fd28-4b95-9f3a-927420aa7db3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:04.328272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:04.410571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:04.411836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:04.412126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:04.412769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] PATH_INFO: `/v3/users/75ef5eae667f4e9e986f191fb5beec36` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:04.415223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ef5eae667f4e9e986f191fb5beec36)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:04.417488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:04.563591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ef5eae667f4e9e986f191fb5beec36'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.567061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ef5eae667f4e9e986f191fb5beec36'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.568197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ef5eae667f4e9e986f191fb5beec36'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.605045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ef5eae667f4e9e986f191fb5beec36'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.624511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ef5eae667f4e9e986f191fb5beec36'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:04.671284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 933/1870] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:04 2022] DELETE /identity/v3/users/75ef5eae667f4e9e986f191fb5beec36 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:04.675632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6633b-cd7e-468a-b236-d203325b0496 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:04.675939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6633b-cd7e-468a-b236-d203325b0496 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:04.676223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d6633b-cd7e-468a-b236-d203325b0496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:04.727598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-16d6633b-cd7e-468a-b236-d203325b0496 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:04.728859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-16d6633b-cd7e-468a-b236-d203325b0496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:04.845547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 938/1871] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:04.983598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:04.985428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:05.224436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff883b-88e9-4ca4-948c-3e8531fd6242 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:05.235428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:05.237016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:05.237016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:05.237016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:05.249259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:05.257708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:05.276091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 934/1872] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:05.911240 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6633b-cd7e-468a-b236-d203325b0496 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:05.919835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-16d6633b-cd7e-468a-b236-d203325b0496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:05.981270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:06.115712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d6633b-cd7e-468a-b236-d203325b0496 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:06.132242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:06.134321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:06.135476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:06.136776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:06.142049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:06.181986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:06.185067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:06.221099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 939/1873] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:06.713324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:06.722660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:06.755833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda9014-4721-4461-869e-eca035578aaa service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:06.763054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:06.765614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:06.766215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:06.766870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] PATH_INFO: `/v3/projects/59bf1fcbe8624af68647407d0fca1bcc` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:06.769223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bf1fcbe8624af68647407d0fca1bcc)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:06.771568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:06.979979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:06.991707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bf1fcbe8624af68647407d0fca1bcc', 'request_id': 'req-cbb83db2-b271-436e-9e78-b1ebeb1e5490'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:06.993058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:06.995600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:07.009332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bf1fcbe8624af68647407d0fca1bcc', 'request_id': 'req-cbb83db2-b271-436e-9e78-b1ebeb1e5490'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.020334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9370-f8be-4073-ad9f-241de1011d80 service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:07.025722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bf1fcbe8624af68647407d0fca1bcc', 'request_id': 'req-cbb83db2-b271-436e-9e78-b1ebeb1e5490'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.064476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] The token cache is being invalidate because project 59bf1fcbe8624af68647407d0fca1bcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:07.064926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.065818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cbb83db2-b271-436e-9e78-b1ebeb1e5490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.067598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 935/1874] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:06 2022] DELETE /identity/v3/projects/59bf1fcbe8624af68647407d0fca1bcc => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:07.074161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd6c2b-812a-4687-9518-98c816f03c03 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:07.074598 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd6c2b-812a-4687-9518-98c816f03c03 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:07.074984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd6c2b-812a-4687-9518-98c816f03c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:07.118437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-affd6c2b-812a-4687-9518-98c816f03c03 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:07.123372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-affd6c2b-812a-4687-9518-98c816f03c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:07.165513 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:07.166666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:07.168720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:07.169228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] PATH_INFO: `/v3/projects/16fe1da4ddb845caa8fab0c2b7c46adc` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:07.173549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16fe1da4ddb845caa8fab0c2b7c46adc)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:07.179475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:07.280475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 936/1875] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:07.281802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd6c2b-812a-4687-9518-98c816f03c03 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:07.283491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-affd6c2b-812a-4687-9518-98c816f03c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:07.301606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16fe1da4ddb845caa8fab0c2b7c46adc', 'request_id': 'req-186c9990-45d9-4f4f-8ac5-7610a2718626'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.315124 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd6c2b-812a-4687-9518-98c816f03c03 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:07.318077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16fe1da4ddb845caa8fab0c2b7c46adc', 'request_id': 'req-186c9990-45d9-4f4f-8ac5-7610a2718626'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.330421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16fe1da4ddb845caa8fab0c2b7c46adc', 'request_id': 'req-186c9990-45d9-4f4f-8ac5-7610a2718626'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.387130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] The token cache is being invalidate because project 16fe1da4ddb845caa8fab0c2b7c46adc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:07.389410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.393616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-186c9990-45d9-4f4f-8ac5-7610a2718626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.401646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 940/1876] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:06 2022] DELETE /identity/v3/projects/16fe1da4ddb845caa8fab0c2b7c46adc => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:07.403495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399624a2-4255-468a-b3b1-51b8f47ce7eb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:07.404000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399624a2-4255-468a-b3b1-51b8f47ce7eb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:07.404465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399624a2-4255-468a-b3b1-51b8f47ce7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:07.439818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-399624a2-4255-468a-b3b1-51b8f47ce7eb None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:07.441564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-399624a2-4255-468a-b3b1-51b8f47ce7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:07.475381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:07.477023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:07.477342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:07.477627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] PATH_INFO: `/v3/users/ce5009b738ff4cff8b0d143ea0049756` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:07.478976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5009b738ff4cff8b0d143ea0049756)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:07.481754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:07.554099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:07.604557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5009b738ff4cff8b0d143ea0049756'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.607140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5009b738ff4cff8b0d143ea0049756'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.608277 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5009b738ff4cff8b0d143ea0049756'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.623629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 941/1877] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:07.627727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399624a2-4255-468a-b3b1-51b8f47ce7eb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:07.642969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5009b738ff4cff8b0d143ea0049756'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.644072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-399624a2-4255-468a-b3b1-51b8f47ce7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:07.663662 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5009b738ff4cff8b0d143ea0049756'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.670145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:07.676155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399624a2-4255-468a-b3b1-51b8f47ce7eb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:07.703739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 937/1878] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:07 2022] DELETE /identity/v3/users/ce5009b738ff4cff8b0d143ea0049756 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:07.703931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:07.704037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:07.704117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:07.704117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:07.704117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] PATH_INFO: `/v3/users/034054c98dee4e0e947063e9726c8322` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:07.707216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=034054c98dee4e0e947063e9726c8322)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:07.713647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:07.834227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb26c31-f61d-46ae-bb0a-9371404ea530 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:07.838795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034054c98dee4e0e947063e9726c8322'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.855528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d04cd-a589-4afc-93d6-fee91bc90625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:07.856958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d04cd-a589-4afc-93d6-fee91bc90625 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:07.857499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d04cd-a589-4afc-93d6-fee91bc90625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:07.857797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d04cd-a589-4afc-93d6-fee91bc90625 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:07.859153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d04cd-a589-4afc-93d6-fee91bc90625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:07.859631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d04cd-a589-4afc-93d6-fee91bc90625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:07.862339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d04cd-a589-4afc-93d6-fee91bc90625 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:07.877262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034054c98dee4e0e947063e9726c8322'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.879042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034054c98dee4e0e947063e9726c8322'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.888687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 938/1879] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:07.891131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b19e72-9b75-4e36-8382-c29b224d5cef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:07.896483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034054c98dee4e0e947063e9726c8322'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.897487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b19e72-9b75-4e36-8382-c29b224d5cef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:07.898185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b19e72-9b75-4e36-8382-c29b224d5cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:07.916599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034054c98dee4e0e947063e9726c8322'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:07.927561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 942/1880] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:07 2022] DELETE /identity/v3/users/034054c98dee4e0e947063e9726c8322 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:07.928635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:07.931248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:07.973955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-73b19e72-9b75-4e36-8382-c29b224d5cef None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:07.975349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-73b19e72-9b75-4e36-8382-c29b224d5cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:08.028721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c13c993-f49f-4b20-8be6-238c7b796f5e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.034330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22526b62-7167-4fbd-af5a-a8caef21b865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.035421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526b62-7167-4fbd-af5a-a8caef21b865 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.035421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526b62-7167-4fbd-af5a-a8caef21b865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.035675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22526b62-7167-4fbd-af5a-a8caef21b865 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.036726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22526b62-7167-4fbd-af5a-a8caef21b865 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.038808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22526b62-7167-4fbd-af5a-a8caef21b865 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.078761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 943/1881] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:07 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:08.082549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22526b62-7167-4fbd-af5a-a8caef21b865 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.119463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 939/1882] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:08.131098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b19e72-9b75-4e36-8382-c29b224d5cef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.133462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-73b19e72-9b75-4e36-8382-c29b224d5cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:08.151530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:08.195678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22526b62-7167-4fbd-af5a-a8caef21b865 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.201698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9680c0-88a6-4d96-9d45-89083669c46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.202423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9680c0-88a6-4d96-9d45-89083669c46a admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.202712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9680c0-88a6-4d96-9d45-89083669c46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.202991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9680c0-88a6-4d96-9d45-89083669c46a admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.211773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9680c0-88a6-4d96-9d45-89083669c46a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.213826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9680c0-88a6-4d96-9d45-89083669c46a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.245658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 944/1883] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:08 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:23:08.248352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9680c0-88a6-4d96-9d45-89083669c46a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.257438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b19e72-9b75-4e36-8382-c29b224d5cef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.294538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9680c0-88a6-4d96-9d45-89083669c46a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.308320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.309094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.310556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.310556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.311208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.312829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.335698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 945/1884] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:08.337084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.339764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:08.356018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:08.357598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cda019-6be0-4bee-b130-d02dda0de3c4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.364095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.364990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.365308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.365589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] PATH_INFO: `/v3/projects/3146fbcf250742c7b9232da34a6a94de` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.366914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3146fbcf250742c7b9232da34a6a94de)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.369294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.373342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4b26b-c40b-4fd5-81a0-26a358487d5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.374307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4b26b-c40b-4fd5-81a0-26a358487d5b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.374813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4b26b-c40b-4fd5-81a0-26a358487d5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.375276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4b26b-c40b-4fd5-81a0-26a358487d5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.388260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4b26b-c40b-4fd5-81a0-26a358487d5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.391279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4b26b-c40b-4fd5-81a0-26a358487d5b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.407475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 940/1885] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:08.408162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4b26b-c40b-4fd5-81a0-26a358487d5b service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.420093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4b26b-c40b-4fd5-81a0-26a358487d5b service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.439049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3146fbcf250742c7b9232da34a6a94de', 'request_id': 'req-7a4a4912-c2fa-481c-9805-49caafe1d71b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.452656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3146fbcf250742c7b9232da34a6a94de', 'request_id': 'req-7a4a4912-c2fa-481c-9805-49caafe1d71b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.463377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3146fbcf250742c7b9232da34a6a94de', 'request_id': 'req-7a4a4912-c2fa-481c-9805-49caafe1d71b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.484753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] The token cache is being invalidate because project 3146fbcf250742c7b9232da34a6a94de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:08.485404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.486669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.492128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 946/1886] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:08 2022] DELETE /identity/v3/projects/3146fbcf250742c7b9232da34a6a94de => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:08.494195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.496520 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:08.505802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.506500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.506965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.507240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.508222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.515812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.538049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 941/1887] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:08.538949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.541080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:08.558570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b1d28-497c-4fd3-a870-af480618e256 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.566332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.567095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.567380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.567661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] PATH_INFO: `/v3/projects/acaf852fa3b041aeb42927629894b678/users/573b7fe0ff054b299a5bfbb7d8eee6ee/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.587578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acaf852fa3b041aeb42927629894b678, user_id=573b7fe0ff054b299a5bfbb7d8eee6ee, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.591571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.624179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 942/1888] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:08 2022] PUT /identity/v3/projects/acaf852fa3b041aeb42927629894b678/users/573b7fe0ff054b299a5bfbb7d8eee6ee/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:08.638150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:08.640968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:08.669306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a4912-c2fa-481c-9805-49caafe1d71b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.737419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282b422-874b-4d93-90dd-44a4ffb84894 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:08.745707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.746856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.746997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.747320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] PATH_INFO: `/v3/projects/381da679be7a4c7ea65a27a926c88eeb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.748853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=381da679be7a4c7ea65a27a926c88eeb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.751067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.777402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:08.809346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:08.811723 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.812200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.812558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] PATH_INFO: `/v3/users/ea71cda5df934fc791c3fa6be9cac60c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.814521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea71cda5df934fc791c3fa6be9cac60c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:08.817047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:08.829313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '381da679be7a4c7ea65a27a926c88eeb', 'request_id': 'req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.845839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '381da679be7a4c7ea65a27a926c88eeb', 'request_id': 'req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.857458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:08.870491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '381da679be7a4c7ea65a27a926c88eeb', 'request_id': 'req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.893062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] The token cache is being invalidate because project 381da679be7a4c7ea65a27a926c88eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:08.893709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.894081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea71cda5df934fc791c3fa6be9cac60c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.894892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-21dbdef5-96f0-48fd-ade5-170bceaa4b4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.897169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 943/1889] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:08 2022] DELETE /identity/v3/projects/381da679be7a4c7ea65a27a926c88eeb => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:08.899413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27f854-8c56-49e8-8b03-d8a1626355f9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.901135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27f854-8c56-49e8-8b03-d8a1626355f9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.901135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27f854-8c56-49e8-8b03-d8a1626355f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.917906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea71cda5df934fc791c3fa6be9cac60c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.918703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea71cda5df934fc791c3fa6be9cac60c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.928916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea71cda5df934fc791c3fa6be9cac60c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.939748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1f56fe68-0ec7-405e-a1af-4cecec935f2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea71cda5df934fc791c3fa6be9cac60c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:08.946442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6f27f854-8c56-49e8-8b03-d8a1626355f9 None None] MFA Rules not processed for user `573b7fe0ff054b299a5bfbb7d8eee6ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:08.948342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f27f854-8c56-49e8-8b03-d8a1626355f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:08.949927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 947/1890] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:08 2022] DELETE /identity/v3/users/ea71cda5df934fc791c3fa6be9cac60c => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:08.951824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af8f1c-9d0c-4d99-b3f0-9c589c0426ab None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:08.952114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af8f1c-9d0c-4d99-b3f0-9c589c0426ab None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:08.952445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af8f1c-9d0c-4d99-b3f0-9c589c0426ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:08.998567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b9af8f1c-9d0c-4d99-b3f0-9c589c0426ab None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:09.000191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b9af8f1c-9d0c-4d99-b3f0-9c589c0426ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.090154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 944/1891] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:09.090997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27f854-8c56-49e8-8b03-d8a1626355f9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.092546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f27f854-8c56-49e8-8b03-d8a1626355f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.121034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 948/1892] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:09.122372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af8f1c-9d0c-4d99-b3f0-9c589c0426ab None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.134020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b9af8f1c-9d0c-4d99-b3f0-9c589c0426ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.150535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af8f1c-9d0c-4d99-b3f0-9c589c0426ab None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.161916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d7a23-c41a-4697-b533-1dfb5a4a7e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.174583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7a23-c41a-4697-b533-1dfb5a4a7e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.176624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7a23-c41a-4697-b533-1dfb5a4a7e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.177112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d7a23-c41a-4697-b533-1dfb5a4a7e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.178550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d7a23-c41a-4697-b533-1dfb5a4a7e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.184073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d7a23-c41a-4697-b533-1dfb5a4a7e00 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.186293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27f854-8c56-49e8-8b03-d8a1626355f9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.191343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.192267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.192554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.192833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] PATH_INFO: `/v3/projects/7b0cc6ed0d284d9e9c8a2d56a556ed04` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.194118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b0cc6ed0d284d9e9c8a2d56a556ed04)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.196471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.228199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 949/1893] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:09 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:09.229969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c43b9a-0ec8-45bd-b4f9-eac49d0b908e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.230408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c43b9a-0ec8-45bd-b4f9-eac49d0b908e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.231025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c43b9a-0ec8-45bd-b4f9-eac49d0b908e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.254477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e1c43b9a-0ec8-45bd-b4f9-eac49d0b908e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:09.255823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e1c43b9a-0ec8-45bd-b4f9-eac49d0b908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.272708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b0cc6ed0d284d9e9c8a2d56a556ed04', 'request_id': 'req-8cafcf45-73dc-4713-961c-443dd9855848'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.285613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b0cc6ed0d284d9e9c8a2d56a556ed04', 'request_id': 'req-8cafcf45-73dc-4713-961c-443dd9855848'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.299444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b0cc6ed0d284d9e9c8a2d56a556ed04', 'request_id': 'req-8cafcf45-73dc-4713-961c-443dd9855848'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.308781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:09.318027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] The token cache is being invalidate because project 7b0cc6ed0d284d9e9c8a2d56a556ed04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:09.318516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.319411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.321951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 945/1894] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:09 2022] DELETE /identity/v3/projects/7b0cc6ed0d284d9e9c8a2d56a556ed04 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:09.322722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.324110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.336034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:09.357700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 950/1895] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:09.358872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c43b9a-0ec8-45bd-b4f9-eac49d0b908e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.360417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e1c43b9a-0ec8-45bd-b4f9-eac49d0b908e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.385469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c43b9a-0ec8-45bd-b4f9-eac49d0b908e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.390413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.391337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.391619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.391895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] PATH_INFO: `/v3/users/8275dbd39af2404482029bf180bf3796` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.393156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8275dbd39af2404482029bf180bf3796)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.395098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.413875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafcf45-73dc-4713-961c-443dd9855848 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.420065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.420865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.421262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.422098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.423801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.426295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.451346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 946/1896] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:09 2022] POST /identity/v3/users => generated 295 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:09.457934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.459273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8275dbd39af2404482029bf180bf3796'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.459829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.473828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038c0a7-20f2-401b-a357-c9c3e3236a19 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.481166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8275dbd39af2404482029bf180bf3796'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.485312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8275dbd39af2404482029bf180bf3796'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.485463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.485463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.485463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.485463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] PATH_INFO: `/v3/projects/4fafcf8cd87846dcb8972c7861146706` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.485752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fafcf8cd87846dcb8972c7861146706)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.486854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.499501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8275dbd39af2404482029bf180bf3796'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.508547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8275dbd39af2404482029bf180bf3796'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.519905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 951/1897] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:09 2022] DELETE /identity/v3/users/8275dbd39af2404482029bf180bf3796 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:09.521195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.529691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.565120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fafcf8cd87846dcb8972c7861146706', 'request_id': 'req-2c74f06c-c5b0-43de-b29b-4280169594b3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.573355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fafcf8cd87846dcb8972c7861146706', 'request_id': 'req-2c74f06c-c5b0-43de-b29b-4280169594b3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.585574 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fafcf8cd87846dcb8972c7861146706', 'request_id': 'req-2c74f06c-c5b0-43de-b29b-4280169594b3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.600888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] The token cache is being invalidate because project 4fafcf8cd87846dcb8972c7861146706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:09.601245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.602253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:09.604796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 947/1898] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:09 2022] DELETE /identity/v3/projects/4fafcf8cd87846dcb8972c7861146706 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:09.605496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.607510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.629330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a22d66-0583-4f2a-ae12-1a49a7bb6479 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.695374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c74f06c-c5b0-43de-b29b-4280169594b3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.703241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a45d35-cf64-4782-b889-a4976706b1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.703947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a45d35-cf64-4782-b889-a4976706b1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.704634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a45d35-cf64-4782-b889-a4976706b1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.705389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a45d35-cf64-4782-b889-a4976706b1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.706177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a45d35-cf64-4782-b889-a4976706b1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.707886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a45d35-cf64-4782-b889-a4976706b1c8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.715325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.716018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.716299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.716572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.717357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:09.717664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.719290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.720056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 948/1899] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:09.721803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ff87-6fe1-4063-a7fe-8ad05be28769 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.722076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:09.722252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ff87-6fe1-4063-a7fe-8ad05be28769 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.722377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ff87-6fe1-4063-a7fe-8ad05be28769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.739928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 952/1900] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:09.739928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.746749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dd86ff87-6fe1-4063-a7fe-8ad05be28769 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:09.747988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dd86ff87-6fe1-4063-a7fe-8ad05be28769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.749550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58526f23-529e-458d-a804-13dc69491486 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.754260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac56ed5-ba8b-402b-b527-3f69c16c07cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.755243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56ed5-ba8b-402b-b527-3f69c16c07cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.755456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56ed5-ba8b-402b-b527-3f69c16c07cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.755732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56ed5-ba8b-402b-b527-3f69c16c07cf admin admin] PATH_INFO: `/v3/projects/7a7f85339b3d4dd9aa8b4a5274d5bd3c/users/e8b1819ec2dd4306aa0584f6de14074a/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.756641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 949/1901] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:09.764450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ff87-6fe1-4063-a7fe-8ad05be28769 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.772565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dd86ff87-6fe1-4063-a7fe-8ad05be28769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.784786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ff87-6fe1-4063-a7fe-8ad05be28769 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.788262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac56ed5-ba8b-402b-b527-3f69c16c07cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7f85339b3d4dd9aa8b4a5274d5bd3c, user_id=e8b1819ec2dd4306aa0584f6de14074a, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.789656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269f88-440e-4102-81b2-5515f09262d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.790426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269f88-440e-4102-81b2-5515f09262d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.790717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269f88-440e-4102-81b2-5515f09262d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.791001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269f88-440e-4102-81b2-5515f09262d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.791497 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac56ed5-ba8b-402b-b527-3f69c16c07cf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.791745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4269f88-440e-4102-81b2-5515f09262d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.793736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4269f88-440e-4102-81b2-5515f09262d0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:09.798594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:09.811876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 953/1902] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:09 2022] PUT /identity/v3/projects/7a7f85339b3d4dd9aa8b4a5274d5bd3c/users/e8b1819ec2dd4306aa0584f6de14074a/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:09.813651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb70bc-9965-4b79-af35-452daa1703ea None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.813943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb70bc-9965-4b79-af35-452daa1703ea None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.814266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb70bc-9965-4b79-af35-452daa1703ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.835457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 950/1903] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:09 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:09.835457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269f88-440e-4102-81b2-5515f09262d0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.836358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6afb70bc-9965-4b79-af35-452daa1703ea None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:09.837640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6afb70bc-9965-4b79-af35-452daa1703ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:09.973215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 954/1904] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:09.974084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb70bc-9965-4b79-af35-452daa1703ea None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:09.976368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269f88-440e-4102-81b2-5515f09262d0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.987693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb70bc-9965-4b79-af35-452daa1703ea None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:09.990623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:09.991328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:09.992124 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:09.992459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:09.993231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:09.994864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.001442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859de7d0-2ce5-45ac-b932-4f4739ffddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.002273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de7d0-2ce5-45ac-b932-4f4739ffddd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.002623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de7d0-2ce5-45ac-b932-4f4739ffddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.002963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859de7d0-2ce5-45ac-b932-4f4739ffddd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.003981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859de7d0-2ce5-45ac-b932-4f4739ffddd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.006182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859de7d0-2ce5-45ac-b932-4f4739ffddd0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.009798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 951/1905] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:10.010689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.012546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.027877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4665f-613f-47f3-8a14-45f13dc8ab36 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.034435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.035377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.035664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.035937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] PATH_INFO: `/v3/projects/2da941bdf37c4adab2720eda49c6f8bd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.037329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da941bdf37c4adab2720eda49c6f8bd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.039747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.041392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 955/1906] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:09 2022] POST /identity/v3/users => generated 289 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:10.048067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859de7d0-2ce5-45ac-b932-4f4739ffddd0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.062205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859de7d0-2ce5-45ac-b932-4f4739ffddd0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.068238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.069308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.069597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.069882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.070654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:10.071329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.072646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.090231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 956/1907] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:10.091095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.102990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f907d9f-570e-4db8-8130-f0a821fc9698 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.108106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42019b3-c255-45db-906d-4ef13b8fb1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.109005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42019b3-c255-45db-906d-4ef13b8fb1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.109946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42019b3-c255-45db-906d-4ef13b8fb1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.109946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42019b3-c255-45db-906d-4ef13b8fb1ea admin admin] PATH_INFO: `/v3/projects/7a7f85339b3d4dd9aa8b4a5274d5bd3c/users/e8b1819ec2dd4306aa0584f6de14074a/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.111819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42019b3-c255-45db-906d-4ef13b8fb1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7f85339b3d4dd9aa8b4a5274d5bd3c, user_id=e8b1819ec2dd4306aa0584f6de14074a, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.114638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42019b3-c255-45db-906d-4ef13b8fb1ea admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.126263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 957/1908] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:10 2022] PUT /identity/v3/projects/7a7f85339b3d4dd9aa8b4a5274d5bd3c/users/e8b1819ec2dd4306aa0584f6de14074a/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:10.127109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da941bdf37c4adab2720eda49c6f8bd', 'request_id': 'req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.127248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42019b3-c255-45db-906d-4ef13b8fb1ea admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.135323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da941bdf37c4adab2720eda49c6f8bd', 'request_id': 'req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.143605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da941bdf37c4adab2720eda49c6f8bd', 'request_id': 'req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.155749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] The token cache is being invalidate because project 2da941bdf37c4adab2720eda49c6f8bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:10.156075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.156732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.158376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 952/1909] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:10 2022] DELETE /identity/v3/projects/2da941bdf37c4adab2720eda49c6f8bd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:10.159159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.160532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.202765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42019b3-c255-45db-906d-4ef13b8fb1ea admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.207972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb572e-598d-4925-9d1f-38b77022c658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.208664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb572e-598d-4925-9d1f-38b77022c658 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.208942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb572e-598d-4925-9d1f-38b77022c658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.209264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb572e-598d-4925-9d1f-38b77022c658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.210040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb572e-598d-4925-9d1f-38b77022c658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.212316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb572e-598d-4925-9d1f-38b77022c658 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.223204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 958/1910] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:10.224581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76cdfa-936f-41fb-9b90-9af170195e22 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.224875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76cdfa-936f-41fb-9b90-9af170195e22 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.225161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76cdfa-936f-41fb-9b90-9af170195e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.233374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52285ea1-cdad-4f5a-9aab-73a8ccdaa4f1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.241583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.243080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.243369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.243640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.244371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.246684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.260647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:10.266009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-fd76cdfa-936f-41fb-9b90-9af170195e22 None None] MFA Rules not processed for user `e8b1819ec2dd4306aa0584f6de14074a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:10.267578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fd76cdfa-936f-41fb-9b90-9af170195e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.283046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 953/1911] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:10 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:10.283837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.285648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.288579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:10.368423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 959/1912] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:10.369793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76cdfa-936f-41fb-9b90-9af170195e22 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.372226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fd76cdfa-936f-41fb-9b90-9af170195e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.391691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76cdfa-936f-41fb-9b90-9af170195e22 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.393006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7edd4-bdb7-4b0e-b142-fb85813454c4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.398600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d437d-5311-4794-bfc9-42607312e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.400529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d437d-5311-4794-bfc9-42607312e16c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.402545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.403087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.403303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d437d-5311-4794-bfc9-42607312e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.403415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.403633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] PATH_INFO: `/v3/users/51ebf734061848d6834b9ee5d3409b9d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.404269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d437d-5311-4794-bfc9-42607312e16c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.405432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d437d-5311-4794-bfc9-42607312e16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.406605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ebf734061848d6834b9ee5d3409b9d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.407667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d437d-5311-4794-bfc9-42607312e16c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.409339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.427669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 960/1913] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:10.428804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d437d-5311-4794-bfc9-42607312e16c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.453589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d437d-5311-4794-bfc9-42607312e16c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.468341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.474946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.475545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.476171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.479339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.481246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ebf734061848d6834b9ee5d3409b9d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.481951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ebf734061848d6834b9ee5d3409b9d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.483270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ebf734061848d6834b9ee5d3409b9d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.485522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.504771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ebf734061848d6834b9ee5d3409b9d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.520210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ebf734061848d6834b9ee5d3409b9d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:10.542857 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 961/1915] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:10 2022] POST /identity/v3/users => generated 291 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:10.544125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.545506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 954/1915] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:10 2022] DELETE /identity/v3/users/51ebf734061848d6834b9ee5d3409b9d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:10.546936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.549412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.551042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.662886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc490c1-9524-4223-a24e-eb88f8e20ef2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.668835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.669923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.670430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.670720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] PATH_INFO: `/v3/projects/5336d7a5e3bf46fe9dd443a8cf373249/users/5f34f3fad0a44acf9e28108b05af7b16/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.689954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5336d7a5e3bf46fe9dd443a8cf373249, user_id=5f34f3fad0a44acf9e28108b05af7b16, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.692954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.696198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:10.701421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c5b07-23dd-447a-ba33-fd40d7fd9e78 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.731178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 955/1916] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:10 2022] PUT /identity/v3/projects/5336d7a5e3bf46fe9dd443a8cf373249/users/5f34f3fad0a44acf9e28108b05af7b16/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:10.732515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:10.734721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.835565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167409c6-f6f8-44e8-9072-cd400221c894 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:10.841410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454ee64-6c67-486b-b013-04d3fb0a7dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.842167 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454ee64-6c67-486b-b013-04d3fb0a7dca admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.842459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454ee64-6c67-486b-b013-04d3fb0a7dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.842742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454ee64-6c67-486b-b013-04d3fb0a7dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.843496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454ee64-6c67-486b-b013-04d3fb0a7dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.845588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0454ee64-6c67-486b-b013-04d3fb0a7dca admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.845966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:10.846726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.847073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.847417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.855138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.858746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 956/1917] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:10.860107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dce6b3-0e8f-4eb2-829b-48f7acc27f6e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:10.860401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dce6b3-0e8f-4eb2-829b-48f7acc27f6e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:10.860732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dce6b3-0e8f-4eb2-829b-48f7acc27f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:10.894742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-44dce6b3-0e8f-4eb2-829b-48f7acc27f6e None None] MFA Rules not processed for user `5f34f3fad0a44acf9e28108b05af7b16`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:10.897051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44dce6b3-0e8f-4eb2-829b-48f7acc27f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:10.955305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:10.970321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:10.972636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:10.985044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 962/1918] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:10.986154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.000508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60ea9d-828c-4000-8d8a-bcb841d6a5f9 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.005897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b52f7c-62d6-498c-a26b-ee44eb0f8409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.006843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52f7c-62d6-498c-a26b-ee44eb0f8409 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.007349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52f7c-62d6-498c-a26b-ee44eb0f8409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.007896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b52f7c-62d6-498c-a26b-ee44eb0f8409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.008907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b52f7c-62d6-498c-a26b-ee44eb0f8409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.010636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b52f7c-62d6-498c-a26b-ee44eb0f8409 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.027087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 963/1919] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:11.028559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b52f7c-62d6-498c-a26b-ee44eb0f8409 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.042621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b52f7c-62d6-498c-a26b-ee44eb0f8409 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.049506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8472887f-3968-4339-be37-fb904891813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.050597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472887f-3968-4339-be37-fb904891813b admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.050988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472887f-3968-4339-be37-fb904891813b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.051424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8472887f-3968-4339-be37-fb904891813b admin admin] PATH_INFO: `/v3/projects/3dc570ddc93b4f64a98cdd8063b03202/users/f902e7315e5849d48f009865920ec245/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.054984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 957/1920] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:11.055965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dce6b3-0e8f-4eb2-829b-48f7acc27f6e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.072547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472887f-3968-4339-be37-fb904891813b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc570ddc93b4f64a98cdd8063b03202, user_id=f902e7315e5849d48f009865920ec245, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.075675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8472887f-3968-4339-be37-fb904891813b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.082584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dce6b3-0e8f-4eb2-829b-48f7acc27f6e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.101328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5acd9-33a1-4c14-936a-a5a59dc2e3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.102117 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5acd9-33a1-4c14-936a-a5a59dc2e3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.102413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5acd9-33a1-4c14-936a-a5a59dc2e3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.102884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5acd9-33a1-4c14-936a-a5a59dc2e3e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.103652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5acd9-33a1-4c14-936a-a5a59dc2e3e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.106431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5acd9-33a1-4c14-936a-a5a59dc2e3e4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.113985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 964/1921] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:11 2022] PUT /identity/v3/projects/3dc570ddc93b4f64a98cdd8063b03202/users/f902e7315e5849d48f009865920ec245/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:11.117782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6659f87-291a-462c-98c3-5e1c81ce592d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.117782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6659f87-291a-462c-98c3-5e1c81ce592d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.118073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6659f87-291a-462c-98c3-5e1c81ce592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.162453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c6659f87-291a-462c-98c3-5e1c81ce592d None None] MFA Rules not processed for user `f902e7315e5849d48f009865920ec245`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:11.163374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 958/1922] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:11 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:11.164113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c6659f87-291a-462c-98c3-5e1c81ce592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:11.164637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5acd9-33a1-4c14-936a-a5a59dc2e3e4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.291609 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 965/1923] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:11.292728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6659f87-291a-462c-98c3-5e1c81ce592d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.294304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:11.302298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5acd9-33a1-4c14-936a-a5a59dc2e3e4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.314008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6659f87-291a-462c-98c3-5e1c81ce592d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.320066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e734e1-3fbc-4373-9fd7-45142616858f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.320960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e734e1-3fbc-4373-9fd7-45142616858f admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.321831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e734e1-3fbc-4373-9fd7-45142616858f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.322212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e734e1-3fbc-4373-9fd7-45142616858f admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.323026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e734e1-3fbc-4373-9fd7-45142616858f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.323835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5bf84-6b70-46de-aa98-fb2156d2fcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.324698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5bf84-6b70-46de-aa98-fb2156d2fcb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.325129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5bf84-6b70-46de-aa98-fb2156d2fcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.325562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e734e1-3fbc-4373-9fd7-45142616858f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.325765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5bf84-6b70-46de-aa98-fb2156d2fcb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.327210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5bf84-6b70-46de-aa98-fb2156d2fcb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.329356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5bf84-6b70-46de-aa98-fb2156d2fcb0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.379878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 966/1924] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:11 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:11.381680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e734e1-3fbc-4373-9fd7-45142616858f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.388761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 959/1925] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:11 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:11.394580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5bf84-6b70-46de-aa98-fb2156d2fcb0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.420332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5bf84-6b70-46de-aa98-fb2156d2fcb0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.712825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e734e1-3fbc-4373-9fd7-45142616858f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.744634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96324d-2361-4660-a8dd-bfb89cf0bc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.745801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96324d-2361-4660-a8dd-bfb89cf0bc82 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.746302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96324d-2361-4660-a8dd-bfb89cf0bc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.747558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96324d-2361-4660-a8dd-bfb89cf0bc82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.748844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96324d-2361-4660-a8dd-bfb89cf0bc82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.752949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96324d-2361-4660-a8dd-bfb89cf0bc82 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.765960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9850f3-aa05-4550-8d4d-8237d62dc354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.767141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9850f3-aa05-4550-8d4d-8237d62dc354 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.767373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9850f3-aa05-4550-8d4d-8237d62dc354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.767663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9850f3-aa05-4550-8d4d-8237d62dc354 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.768816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9850f3-aa05-4550-8d4d-8237d62dc354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.779063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 960/1926] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:11.781284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96324d-2361-4660-a8dd-bfb89cf0bc82 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.789341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9850f3-aa05-4550-8d4d-8237d62dc354 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.823019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96324d-2361-4660-a8dd-bfb89cf0bc82 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.839103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e828d-c38e-4d5c-a64e-fd247eb2e0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.840019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e828d-c38e-4d5c-a64e-fd247eb2e0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.840144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e828d-c38e-4d5c-a64e-fd247eb2e0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.840530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e828d-c38e-4d5c-a64e-fd247eb2e0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.841450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e828d-c38e-4d5c-a64e-fd247eb2e0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.843110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e828d-c38e-4d5c-a64e-fd247eb2e0d0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.867138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 961/1927] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:11.868211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e828d-c38e-4d5c-a64e-fd247eb2e0d0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.878583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 967/1928] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:11 2022] POST /identity/v3/users => generated 297 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:11.891639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e828d-c38e-4d5c-a64e-fd247eb2e0d0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.896393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9850f3-aa05-4550-8d4d-8237d62dc354 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:11.906448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e63eef-71f1-4479-85ee-4e424ea5a551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.907379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63eef-71f1-4479-85ee-4e424ea5a551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.907663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63eef-71f1-4479-85ee-4e424ea5a551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.907943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e63eef-71f1-4479-85ee-4e424ea5a551 admin admin] PATH_INFO: `/v3/projects/e3dd833359504229bc9b4e29a421a440/users/f7ab42130d68466ca814db43f62a1d63/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.940919 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e63eef-71f1-4479-85ee-4e424ea5a551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3dd833359504229bc9b4e29a421a440, user_id=f7ab42130d68466ca814db43f62a1d63, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.949043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e63eef-71f1-4479-85ee-4e424ea5a551 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.956464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9850f3-aa05-4550-8d4d-8237d62dc354 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:11.969426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27d9c7-3442-4b2d-b547-ddf7fe298a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:11.970972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d9c7-3442-4b2d-b547-ddf7fe298a84 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.971108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 962/1929] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:11 2022] PUT /identity/v3/projects/e3dd833359504229bc9b4e29a421a440/users/f7ab42130d68466ca814db43f62a1d63/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:11.971191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d9c7-3442-4b2d-b547-ddf7fe298a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.973346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d9c7-3442-4b2d-b547-ddf7fe298a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.974487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27d9c7-3442-4b2d-b547-ddf7fe298a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:11.976945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27d9c7-3442-4b2d-b547-ddf7fe298a84 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:11.992478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ed0d-e396-4550-9b72-f60013131680 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:11.992478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ed0d-e396-4550-9b72-f60013131680 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:11.992478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b7ed0d-e396-4550-9b72-f60013131680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:11.998716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 968/1930] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:12.003099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27d9c7-3442-4b2d-b547-ddf7fe298a84 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.010542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:12.020699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27d9c7-3442-4b2d-b547-ddf7fe298a84 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.023903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-88b7ed0d-e396-4550-9b72-f60013131680 None None] MFA Rules not processed for user `f7ab42130d68466ca814db43f62a1d63`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:12.025217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-88b7ed0d-e396-4550-9b72-f60013131680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:12.100043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a96e1-dea8-4ace-8b84-2856dbb261e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.101125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a96e1-dea8-4ace-8b84-2856dbb261e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.101762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a96e1-dea8-4ace-8b84-2856dbb261e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.102508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a96e1-dea8-4ace-8b84-2856dbb261e2 admin admin] PATH_INFO: `/v3/projects/20598005537949149c827c70ca7a06dd/users/c9740b81a0754af99f9f5280632cfe4b/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.107670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 963/1931] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:12.112209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7ed0d-e396-4550-9b72-f60013131680 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.123319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a96e1-dea8-4ace-8b84-2856dbb261e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20598005537949149c827c70ca7a06dd, user_id=c9740b81a0754af99f9f5280632cfe4b, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.123941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b7ed0d-e396-4550-9b72-f60013131680 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.127040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a96e1-dea8-4ace-8b84-2856dbb261e2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.129006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718403e-e989-4bec-a80c-aa9250b6d922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.129817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718403e-e989-4bec-a80c-aa9250b6d922 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.130171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718403e-e989-4bec-a80c-aa9250b6d922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.130583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718403e-e989-4bec-a80c-aa9250b6d922 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.131312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718403e-e989-4bec-a80c-aa9250b6d922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.133460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718403e-e989-4bec-a80c-aa9250b6d922 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.144560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 969/1932] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:12 2022] PUT /identity/v3/projects/20598005537949149c827c70ca7a06dd/users/c9740b81a0754af99f9f5280632cfe4b/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:12.148499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a96e1-dea8-4ace-8b84-2856dbb261e2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.168166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 964/1933] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:12 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:23:12.169352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718403e-e989-4bec-a80c-aa9250b6d922 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.190868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:12.227579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a96e1-dea8-4ace-8b84-2856dbb261e2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.258711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718403e-e989-4bec-a80c-aa9250b6d922 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.264308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7feb6-1f75-410b-b06b-fe1c62b94ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.265507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7feb6-1f75-410b-b06b-fe1c62b94ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.265660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7feb6-1f75-410b-b06b-fe1c62b94ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.266382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7feb6-1f75-410b-b06b-fe1c62b94ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.267091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7feb6-1f75-410b-b06b-fe1c62b94ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.269067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7feb6-1f75-410b-b06b-fe1c62b94ba1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.294541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 965/1934] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:12 2022] POST /identity/v3/users => generated 295 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:12.298901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7feb6-1f75-410b-b06b-fe1c62b94ba1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.299886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2864e-ef83-464e-8b16-f362954abfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.300953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2864e-ef83-464e-8b16-f362954abfed admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.301560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2864e-ef83-464e-8b16-f362954abfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.302127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2864e-ef83-464e-8b16-f362954abfed admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.303023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2864e-ef83-464e-8b16-f362954abfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.305089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2864e-ef83-464e-8b16-f362954abfed admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.312850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7feb6-1f75-410b-b06b-fe1c62b94ba1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.320591 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 970/1935] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:23:12.321125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640e07b-37af-4b20-a216-e3960582194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.321125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640e07b-37af-4b20-a216-e3960582194a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.321125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640e07b-37af-4b20-a216-e3960582194a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.321125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640e07b-37af-4b20-a216-e3960582194a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.321514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640e07b-37af-4b20-a216-e3960582194a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.321959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2864e-ef83-464e-8b16-f362954abfed admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.322317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2640e07b-37af-4b20-a216-e3960582194a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.333123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2864e-ef83-464e-8b16-f362954abfed admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.334311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 966/1936] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:12.338422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640e07b-37af-4b20-a216-e3960582194a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.341661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f3676-7784-4971-824d-7765c96f252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.342624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f3676-7784-4971-824d-7765c96f252b admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.342907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f3676-7784-4971-824d-7765c96f252b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.343178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f3676-7784-4971-824d-7765c96f252b admin admin] PATH_INFO: `/v3/projects/20598005537949149c827c70ca7a06dd/users/c9740b81a0754af99f9f5280632cfe4b/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.345302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f3676-7784-4971-824d-7765c96f252b admin admin] RBAC: Authorizing `identity:create_grant(project_id=20598005537949149c827c70ca7a06dd, user_id=c9740b81a0754af99f9f5280632cfe4b, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.348096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f3676-7784-4971-824d-7765c96f252b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.350358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640e07b-37af-4b20-a216-e3960582194a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.356433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.357452 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.357851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.358304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] PATH_INFO: `/v3/projects/df85f6946e424a6aa1dd55bf029bfe3c/users/3e7d6b08b9384a79b836716527adec08/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.360932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 971/1937] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:12 2022] PUT /identity/v3/projects/20598005537949149c827c70ca7a06dd/users/c9740b81a0754af99f9f5280632cfe4b/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:12.373304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae4f06-a85a-4058-a0fc-a477bf61c91b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.373593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae4f06-a85a-4058-a0fc-a477bf61c91b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.373868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae4f06-a85a-4058-a0fc-a477bf61c91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.386270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df85f6946e424a6aa1dd55bf029bfe3c, user_id=3e7d6b08b9384a79b836716527adec08, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.389242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.411379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5bae4f06-a85a-4058-a0fc-a477bf61c91b None None] MFA Rules not processed for user `c9740b81a0754af99f9f5280632cfe4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:12.413393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5bae4f06-a85a-4058-a0fc-a477bf61c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:12.413966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 967/1938] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:12 2022] PUT /identity/v3/projects/df85f6946e424a6aa1dd55bf029bfe3c/users/3e7d6b08b9384a79b836716527adec08/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:12.415063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.416670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:12.418544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:12.517629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d655fa-850b-437a-a926-eabf0b114005 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.523561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.524377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.524900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.525774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] PATH_INFO: `/v3/projects/927236d7cbc048f68e805499cafc0f52` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.527254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=927236d7cbc048f68e805499cafc0f52)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.531245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.536524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 972/1939] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:12.542872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae4f06-a85a-4058-a0fc-a477bf61c91b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.559848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae4f06-a85a-4058-a0fc-a477bf61c91b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.566947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a318f-9dea-4461-9aea-dac76947435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.567972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a318f-9dea-4461-9aea-dac76947435d admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.568457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a318f-9dea-4461-9aea-dac76947435d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.569070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a318f-9dea-4461-9aea-dac76947435d admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.570541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a318f-9dea-4461-9aea-dac76947435d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.573813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a318f-9dea-4461-9aea-dac76947435d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.589102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 973/1940] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:12.590752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a318f-9dea-4461-9aea-dac76947435d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.649511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927236d7cbc048f68e805499cafc0f52', 'request_id': 'req-78c09f22-a235-4e56-b12f-196044791b0f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:12.657523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927236d7cbc048f68e805499cafc0f52', 'request_id': 'req-78c09f22-a235-4e56-b12f-196044791b0f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:12.668514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927236d7cbc048f68e805499cafc0f52', 'request_id': 'req-78c09f22-a235-4e56-b12f-196044791b0f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:12.685826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:12.694388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] The token cache is being invalidate because project 927236d7cbc048f68e805499cafc0f52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:12.694733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:12.695590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:12.702936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 968/1941] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:12 2022] DELETE /identity/v3/projects/927236d7cbc048f68e805499cafc0f52 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:12.702936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:12.705944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:12.714512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a318f-9dea-4461-9aea-dac76947435d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.849599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c09f22-a235-4e56-b12f-196044791b0f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:12.858782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb105903-f64c-4a02-8c54-05e80f1aa250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.859727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb105903-f64c-4a02-8c54-05e80f1aa250 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.860104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb105903-f64c-4a02-8c54-05e80f1aa250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.860476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb105903-f64c-4a02-8c54-05e80f1aa250 admin admin] PATH_INFO: `/v3/projects/df85f6946e424a6aa1dd55bf029bfe3c/users/3e7d6b08b9384a79b836716527adec08/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.862968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb105903-f64c-4a02-8c54-05e80f1aa250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df85f6946e424a6aa1dd55bf029bfe3c, user_id=3e7d6b08b9384a79b836716527adec08, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:12.865873 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb105903-f64c-4a02-8c54-05e80f1aa250 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:12.889085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 969/1942] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:12 2022] PUT /identity/v3/projects/df85f6946e424a6aa1dd55bf029bfe3c/users/3e7d6b08b9384a79b836716527adec08/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:12.889085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f46ad-033c-4d6f-a2f2-5598345b9b11 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.889085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f46ad-033c-4d6f-a2f2-5598345b9b11 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.889085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f46ad-033c-4d6f-a2f2-5598345b9b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.895218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c087c-d4df-4bbd-9b44-34d815e61d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:12.896136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c087c-d4df-4bbd-9b44-34d815e61d8b service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:12.896513 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c087c-d4df-4bbd-9b44-34d815e61d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:12.896891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c087c-d4df-4bbd-9b44-34d815e61d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:12.901331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c36c087c-d4df-4bbd-9b44-34d815e61d8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:12.910179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-df8f46ad-033c-4d6f-a2f2-5598345b9b11 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:12.912542 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-df8f46ad-033c-4d6f-a2f2-5598345b9b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:13.003115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 970/1943] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:13.003115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad21d4-2a97-4c58-8113-169c2bcf379c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.003115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad21d4-2a97-4c58-8113-169c2bcf379c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.003115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad21d4-2a97-4c58-8113-169c2bcf379c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.037360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c5ad21d4-2a97-4c58-8113-169c2bcf379c None None] MFA Rules not processed for user `3e7d6b08b9384a79b836716527adec08`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:13.038819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c5ad21d4-2a97-4c58-8113-169c2bcf379c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:13.058401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:13.063039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c087c-d4df-4bbd-9b44-34d815e61d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.064820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c087c-d4df-4bbd-9b44-34d815e61d8b service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.080423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 974/1944] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:13.082302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb5ff-94bb-4729-bcd0-f484df3dfcfb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.082593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb5ff-94bb-4729-bcd0-f484df3dfcfb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.082876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111fb5ff-94bb-4729-bcd0-f484df3dfcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.207775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-111fb5ff-94bb-4729-bcd0-f484df3dfcfb None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:13.209629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-111fb5ff-94bb-4729-bcd0-f484df3dfcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:13.233158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 971/1945] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:13.241840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 975/1946] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:13.247128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad21d4-2a97-4c58-8113-169c2bcf379c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.260590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fb5ff-94bb-4729-bcd0-f484df3dfcfb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.263101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad21d4-2a97-4c58-8113-169c2bcf379c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.268947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-111fb5ff-94bb-4729-bcd0-f484df3dfcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:13.269531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.270236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.270519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.270851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.271782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:13.272112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.274002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.301101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 972/1947] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:13.306888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.327351 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3eef6-16da-49c5-9cba-ee95d88243e2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.343807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc52ce-c33e-4eeb-aefc-e687b7d12702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.344748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc52ce-c33e-4eeb-aefc-e687b7d12702 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.345196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc52ce-c33e-4eeb-aefc-e687b7d12702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.345752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bc52ce-c33e-4eeb-aefc-e687b7d12702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.346636 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc52ce-c33e-4eeb-aefc-e687b7d12702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.348775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bc52ce-c33e-4eeb-aefc-e687b7d12702 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.353359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:13.407450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111fb5ff-94bb-4729-bcd0-f484df3dfcfb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.412093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.412925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.413165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.413948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.415305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 973/1948] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:13 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:23:13.419923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:13.423509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc52ce-c33e-4eeb-aefc-e687b7d12702 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.568080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bc52ce-c33e-4eeb-aefc-e687b7d12702 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.573755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81498a84-b649-4ae8-9f65-0381eece6227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.574441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81498a84-b649-4ae8-9f65-0381eece6227 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.574863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81498a84-b649-4ae8-9f65-0381eece6227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.575135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81498a84-b649-4ae8-9f65-0381eece6227 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.575847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81498a84-b649-4ae8-9f65-0381eece6227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.577861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81498a84-b649-4ae8-9f65-0381eece6227 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.582439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.584546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.604918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 976/1949] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:13.614619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 974/1950] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:13 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:13.620224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.632021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52753a-333e-476e-b024-b21503d29319 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.638046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a46533-8e88-4534-95e8-9d2041f2cd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.638733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a46533-8e88-4534-95e8-9d2041f2cd8f admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.639019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a46533-8e88-4534-95e8-9d2041f2cd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.639292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a46533-8e88-4534-95e8-9d2041f2cd8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.640029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a46533-8e88-4534-95e8-9d2041f2cd8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.641648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a46533-8e88-4534-95e8-9d2041f2cd8f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.653777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 977/1951] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:23:13.661317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81498a84-b649-4ae8-9f65-0381eece6227 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.680177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81498a84-b649-4ae8-9f65-0381eece6227 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.687760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.688747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.689793 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.690331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.691269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.693077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.704672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 975/1952] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:13.708788 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a46533-8e88-4534-95e8-9d2041f2cd8f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.721777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a46533-8e88-4534-95e8-9d2041f2cd8f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.728683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedc2b7-2d78-4ac7-aefd-d10b44c966c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.729924 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedc2b7-2d78-4ac7-aefd-d10b44c966c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.730271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedc2b7-2d78-4ac7-aefd-d10b44c966c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.730963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedc2b7-2d78-4ac7-aefd-d10b44c966c5 admin admin] PATH_INFO: `/v3/projects/c98413c1705644359db4e3ebe75d8d1f/users/aa2570b7f6684bdf98e7b4fc8982f2fa/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.748024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.749313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedc2b7-2d78-4ac7-aefd-d10b44c966c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c98413c1705644359db4e3ebe75d8d1f, user_id=aa2570b7f6684bdf98e7b4fc8982f2fa, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.750761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:13.752222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedc2b7-2d78-4ac7-aefd-d10b44c966c5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.766378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397eefd-3bb5-4a42-9f53-14d06f03c339 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.768559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 978/1953] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:13 2022] PUT /identity/v3/projects/c98413c1705644359db4e3ebe75d8d1f/users/aa2570b7f6684bdf98e7b4fc8982f2fa/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:13.780479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34720c57-d588-4a6b-8c13-33717c83d66c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.780799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34720c57-d588-4a6b-8c13-33717c83d66c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.781032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34720c57-d588-4a6b-8c13-33717c83d66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.831527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-34720c57-d588-4a6b-8c13-33717c83d66c None None] MFA Rules not processed for user `aa2570b7f6684bdf98e7b4fc8982f2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:13.832866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-34720c57-d588-4a6b-8c13-33717c83d66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:13.854241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.855228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.855510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.856134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] PATH_INFO: `/v3/users/f902e7315e5849d48f009865920ec245` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.857801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f902e7315e5849d48f009865920ec245)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.861201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.906241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:13.917631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 979/1954] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:13.922200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34720c57-d588-4a6b-8c13-33717c83d66c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:13.949695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34720c57-d588-4a6b-8c13-33717c83d66c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:13.955511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:13.956464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:13.957440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:13.957732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:13.958535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f902e7315e5849d48f009865920ec245'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:13.958731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:13.959264 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f902e7315e5849d48f009865920ec245'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:13.959900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f902e7315e5849d48f009865920ec245'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:13.960917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:13.984925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f902e7315e5849d48f009865920ec245'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:13.987091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:14.001123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3cc38abc-c530-434f-8fbc-a60a5125d41b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f902e7315e5849d48f009865920ec245'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.016606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 980/1955] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:13 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:23:14.017812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.021309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:14.037138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 976/1956] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:13 2022] DELETE /identity/v3/users/f902e7315e5849d48f009865920ec245 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:14.042823 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1be76f-4688-45ee-8797-f785e21d8e7b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.043111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1be76f-4688-45ee-8797-f785e21d8e7b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.043477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1be76f-4688-45ee-8797-f785e21d8e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.070946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6e1be76f-4688-45ee-8797-f785e21d8e7b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:14.072297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e1be76f-4688-45ee-8797-f785e21d8e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:14.111366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0eb92-c468-4c5b-a33d-fac5f1a0e774 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.136661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 977/1957] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:14.147432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1be76f-4688-45ee-8797-f785e21d8e7b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.149857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6e1be76f-4688-45ee-8797-f785e21d8e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:14.174757 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759b3a-4ed4-4b5e-bdd7-a9c87e1a7e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.175454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759b3a-4ed4-4b5e-bdd7-a9c87e1a7e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.175741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759b3a-4ed4-4b5e-bdd7-a9c87e1a7e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.176031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76759b3a-4ed4-4b5e-bdd7-a9c87e1a7e4e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.176736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76759b3a-4ed4-4b5e-bdd7-a9c87e1a7e4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.178709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76759b3a-4ed4-4b5e-bdd7-a9c87e1a7e4e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.207190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 981/1958] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:14 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:14.208316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759b3a-4ed4-4b5e-bdd7-a9c87e1a7e4e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.219099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76759b3a-4ed4-4b5e-bdd7-a9c87e1a7e4e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.219683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1be76f-4688-45ee-8797-f785e21d8e7b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.223438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78948d-afca-4352-b28e-e4d5ff26a7d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.223781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad81eb-74ca-40cd-9cac-5fc519321bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.224106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78948d-afca-4352-b28e-e4d5ff26a7d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.224394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78948d-afca-4352-b28e-e4d5ff26a7d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.224498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad81eb-74ca-40cd-9cac-5fc519321bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.224675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78948d-afca-4352-b28e-e4d5ff26a7d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.224772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad81eb-74ca-40cd-9cac-5fc519321bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.225004 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad81eb-74ca-40cd-9cac-5fc519321bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.225820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad81eb-74ca-40cd-9cac-5fc519321bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.227717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad81eb-74ca-40cd-9cac-5fc519321bc8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.235310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78948d-afca-4352-b28e-e4d5ff26a7d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.236787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78948d-afca-4352-b28e-e4d5ff26a7d1 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.242891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 982/1959] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:14.243888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad81eb-74ca-40cd-9cac-5fc519321bc8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.248224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 978/1960] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:14.254108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad81eb-74ca-40cd-9cac-5fc519321bc8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.259399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673ad0-734d-4e81-9b7e-bc1442711cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.260312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673ad0-734d-4e81-9b7e-bc1442711cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.260599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673ad0-734d-4e81-9b7e-bc1442711cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.260882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f673ad0-734d-4e81-9b7e-bc1442711cca admin admin] PATH_INFO: `/v3/projects/8d267d8926d244b8a9fb37781b831df4/users/1356196ef4974fb1a05132b17dbd1f94/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.277583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f673ad0-734d-4e81-9b7e-bc1442711cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d267d8926d244b8a9fb37781b831df4, user_id=1356196ef4974fb1a05132b17dbd1f94, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.280281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f673ad0-734d-4e81-9b7e-bc1442711cca admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.299963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 983/1961] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:14 2022] PUT /identity/v3/projects/8d267d8926d244b8a9fb37781b831df4/users/1356196ef4974fb1a05132b17dbd1f94/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:14.304121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78948d-afca-4352-b28e-e4d5ff26a7d1 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.316362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78948d-afca-4352-b28e-e4d5ff26a7d1 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.391961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc577de1-5cf6-4e1b-a0a0-f85e2b807f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.393085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc577de1-5cf6-4e1b-a0a0-f85e2b807f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.393606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc577de1-5cf6-4e1b-a0a0-f85e2b807f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.394064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc577de1-5cf6-4e1b-a0a0-f85e2b807f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.395119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc577de1-5cf6-4e1b-a0a0-f85e2b807f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.397082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc577de1-5cf6-4e1b-a0a0-f85e2b807f31 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.410267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 979/1962] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:14.416481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673ad0-734d-4e81-9b7e-bc1442711cca admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.439058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f673ad0-734d-4e81-9b7e-bc1442711cca admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.446571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.447593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.447914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.448789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] PATH_INFO: `/v3/projects/8d267d8926d244b8a9fb37781b831df4/users/1356196ef4974fb1a05132b17dbd1f94/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.451006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d267d8926d244b8a9fb37781b831df4, user_id=1356196ef4974fb1a05132b17dbd1f94, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.454332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.466665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 984/1963] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:14 2022] PUT /identity/v3/projects/8d267d8926d244b8a9fb37781b831df4/users/1356196ef4974fb1a05132b17dbd1f94/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:14.478234 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816002f-c9c3-4503-9243-94d8eadeabc7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.478568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816002f-c9c3-4503-9243-94d8eadeabc7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.478809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3816002f-c9c3-4503-9243-94d8eadeabc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.513230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3816002f-c9c3-4503-9243-94d8eadeabc7 None None] MFA Rules not processed for user `1356196ef4974fb1a05132b17dbd1f94`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:14.513230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3816002f-c9c3-4503-9243-94d8eadeabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:14.526472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:14.581356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.583560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:14.626246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:14.633021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 980/1964] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:14.644502 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816002f-c9c3-4503-9243-94d8eadeabc7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.646070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3816002f-c9c3-4503-9243-94d8eadeabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:14.705779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c502157-f0a6-4c40-8a90-901124f2278d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.718440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.719374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.719662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.720096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] PATH_INFO: `/v3/projects/3dc570ddc93b4f64a98cdd8063b03202` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.721442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc570ddc93b4f64a98cdd8063b03202)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.723889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.741464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3816002f-c9c3-4503-9243-94d8eadeabc7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.746330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72697df5-22cf-445e-91ff-4f65c61a3d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.747058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72697df5-22cf-445e-91ff-4f65c61a3d16 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.747340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72697df5-22cf-445e-91ff-4f65c61a3d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.747718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72697df5-22cf-445e-91ff-4f65c61a3d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.758532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72697df5-22cf-445e-91ff-4f65c61a3d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.760262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72697df5-22cf-445e-91ff-4f65c61a3d16 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.774000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 981/1965] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:14.797701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc570ddc93b4f64a98cdd8063b03202', 'request_id': 'req-8f16b4d9-0486-4a6c-b584-e397b82b332b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.808169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc570ddc93b4f64a98cdd8063b03202', 'request_id': 'req-8f16b4d9-0486-4a6c-b584-e397b82b332b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.814973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc570ddc93b4f64a98cdd8063b03202', 'request_id': 'req-8f16b4d9-0486-4a6c-b584-e397b82b332b'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.830154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] The token cache is being invalidate because project 3dc570ddc93b4f64a98cdd8063b03202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:14.830449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.831134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.833230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 985/1966] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:14 2022] DELETE /identity/v3/projects/3dc570ddc93b4f64a98cdd8063b03202 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:14.837412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72697df5-22cf-445e-91ff-4f65c61a3d16 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.850072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72697df5-22cf-445e-91ff-4f65c61a3d16 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.884796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:14.886618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:14.924489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.925426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.925715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.926126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] PATH_INFO: `/v3/users/c9740b81a0754af99f9f5280632cfe4b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.927348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9740b81a0754af99f9f5280632cfe4b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:14.929318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:14.950337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16b4d9-0486-4a6c-b584-e397b82b332b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:14.956907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:14.957895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:14.958240 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:14.958524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:14.961374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:14.980667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9740b81a0754af99f9f5280632cfe4b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.981761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9740b81a0754af99f9f5280632cfe4b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.982593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9740b81a0754af99f9f5280632cfe4b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:14.987469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:14.997375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9740b81a0754af99f9f5280632cfe4b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.006398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9740b81a0754af99f9f5280632cfe4b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.019514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 982/1967] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:14 2022] DELETE /identity/v3/users/c9740b81a0754af99f9f5280632cfe4b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:15.030113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:15.031740 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:15.042357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 986/1968] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:15.377706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:15.379184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:15.431679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:15.434368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:15.454516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb90fa-411a-4cc5-9c8c-0ceafa732541 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:15.460658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:15.477558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55c114f-fb41-4df2-a88c-16473168c234 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:15.482882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:15.483838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.484241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.484364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] PATH_INFO: `/v3/users/aa2570b7f6684bdf98e7b4fc8982f2fa` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.485726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa2570b7f6684bdf98e7b4fc8982f2fa)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:15.487660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:15.530829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:15.531724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.532006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.532289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] PATH_INFO: `/v3/projects/20598005537949149c827c70ca7a06dd` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.533549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20598005537949149c827c70ca7a06dd)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:15.535464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:15.542589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa2570b7f6684bdf98e7b4fc8982f2fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.543345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa2570b7f6684bdf98e7b4fc8982f2fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.544013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa2570b7f6684bdf98e7b4fc8982f2fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.556956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa2570b7f6684bdf98e7b4fc8982f2fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.566103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-eba56e77-73bb-40d8-aad2-7a130b4e9a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa2570b7f6684bdf98e7b4fc8982f2fa'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.583635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 983/1969] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:15 2022] DELETE /identity/v3/users/aa2570b7f6684bdf98e7b4fc8982f2fa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:15.585422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3e5d5-ebdf-49b3-8da8-f8155865deeb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.585715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3e5d5-ebdf-49b3-8da8-f8155865deeb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.585996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3e5d5-ebdf-49b3-8da8-f8155865deeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.590298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20598005537949149c827c70ca7a06dd', 'request_id': 'req-8281bfdf-97e0-434a-a2a3-9d86f622fc45'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.600484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20598005537949149c827c70ca7a06dd', 'request_id': 'req-8281bfdf-97e0-434a-a2a3-9d86f622fc45'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.604714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-34f3e5d5-ebdf-49b3-8da8-f8155865deeb None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:15.606372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-34f3e5d5-ebdf-49b3-8da8-f8155865deeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:15.609088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20598005537949149c827c70ca7a06dd', 'request_id': 'req-8281bfdf-97e0-434a-a2a3-9d86f622fc45'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.621376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] The token cache is being invalidate because project 20598005537949149c827c70ca7a06dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:15.621660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.622325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8281bfdf-97e0-434a-a2a3-9d86f622fc45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:15.623911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 987/1970] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:15 2022] DELETE /identity/v3/projects/20598005537949149c827c70ca7a06dd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:15.628729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b54a-d239-496b-ba3c-035fc3bba0e4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.629018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b54a-d239-496b-ba3c-035fc3bba0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.629362 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd5b54a-d239-496b-ba3c-035fc3bba0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.649683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5bd5b54a-d239-496b-ba3c-035fc3bba0e4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:15.650955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5bd5b54a-d239-496b-ba3c-035fc3bba0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:15.654916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:15.675224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 984/1971] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:15.686128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3e5d5-ebdf-49b3-8da8-f8155865deeb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:15.687708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-34f3e5d5-ebdf-49b3-8da8-f8155865deeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:15.727903 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 988/1972] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:15.736654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0f4bd-ac8e-4e36-a287-a7e9ea53f5c9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.736937 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0f4bd-ac8e-4e36-a287-a7e9ea53f5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.737251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0f4bd-ac8e-4e36-a287-a7e9ea53f5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.757090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-35f0f4bd-ac8e-4e36-a287-a7e9ea53f5c9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:15.758530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-35f0f4bd-ac8e-4e36-a287-a7e9ea53f5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:15.759639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3e5d5-ebdf-49b3-8da8-f8155865deeb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:15.763701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd469cc-383f-4f0b-91eb-9d72e8b97d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:15.764367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd469cc-383f-4f0b-91eb-9d72e8b97d0a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.764648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd469cc-383f-4f0b-91eb-9d72e8b97d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.764922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd469cc-383f-4f0b-91eb-9d72e8b97d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.769811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 989/1973] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:15.777741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0f4bd-ac8e-4e36-a287-a7e9ea53f5c9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:15.778300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd469cc-383f-4f0b-91eb-9d72e8b97d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:15.779744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd469cc-383f-4f0b-91eb-9d72e8b97d0a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:15.787840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0f4bd-ac8e-4e36-a287-a7e9ea53f5c9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:15.790115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 985/1974] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:15.794448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:15.795165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.795454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.795733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.796973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:15.796973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:15.798512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:15.812943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 990/1975] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:23:15.818002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd469cc-383f-4f0b-91eb-9d72e8b97d0a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:15.829266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd469cc-383f-4f0b-91eb-9d72e8b97d0a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:15.835514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5f15b-b9b1-4930-a15e-d765d678a554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:15.837332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5f15b-b9b1-4930-a15e-d765d678a554 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.837332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5f15b-b9b1-4930-a15e-d765d678a554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.837332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5f15b-b9b1-4930-a15e-d765d678a554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.838062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5f15b-b9b1-4930-a15e-d765d678a554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:15.839492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5f15b-b9b1-4930-a15e-d765d678a554 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:15.875634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 986/1976] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:15 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:23:15.877372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:15.890449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776eae8-a1c3-4d88-a52b-1df902d0afdc admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:15.915567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:15.974044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d97782-ff7b-4c45-8eeb-949dfec9cda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:15.975536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d97782-ff7b-4c45-8eeb-949dfec9cda7 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:15.975536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d97782-ff7b-4c45-8eeb-949dfec9cda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:15.975536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d97782-ff7b-4c45-8eeb-949dfec9cda7 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:15.976443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d97782-ff7b-4c45-8eeb-949dfec9cda7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:15.979128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d97782-ff7b-4c45-8eeb-949dfec9cda7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.012039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 991/1977] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:15 2022] POST /identity/v3/users => generated 291 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:16.016955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5f15b-b9b1-4930-a15e-d765d678a554 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.028861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5f15b-b9b1-4930-a15e-d765d678a554 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:16.034394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8e8e4-3500-47b5-b009-d4d3e875e201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:16.035116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8e8e4-3500-47b5-b009-d4d3e875e201 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.035429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8e8e4-3500-47b5-b009-d4d3e875e201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.035776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8e8e4-3500-47b5-b009-d4d3e875e201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.036717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8e8e4-3500-47b5-b009-d4d3e875e201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:16.038243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8e8e4-3500-47b5-b009-d4d3e875e201 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.053860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 987/1978] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:16.058358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d97782-ff7b-4c45-8eeb-949dfec9cda7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.072533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d97782-ff7b-4c45-8eeb-949dfec9cda7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:16.080624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6af8f-cc74-417c-b623-890f3783b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:16.081488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6af8f-cc74-417c-b623-890f3783b3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.081850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6af8f-cc74-417c-b623-890f3783b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.082293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6af8f-cc74-417c-b623-890f3783b3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.083141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6af8f-cc74-417c-b623-890f3783b3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:16.085627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f6af8f-cc74-417c-b623-890f3783b3d8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.106459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8e8e4-3500-47b5-b009-d4d3e875e201 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.106841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 992/1979] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:16.120098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8e8e4-3500-47b5-b009-d4d3e875e201 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:16.126699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:16.129788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.129788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.130038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] PATH_INFO: `/v3/projects/6c47df4422604b42b9fd9ec4b99031aa/users/dd650b1613dc44eca5cc42b88d38fed0/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.151599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c47df4422604b42b9fd9ec4b99031aa, user_id=dd650b1613dc44eca5cc42b88d38fed0, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:16.154656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.181111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 988/1980] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:16 2022] PUT /identity/v3/projects/6c47df4422604b42b9fd9ec4b99031aa/users/dd650b1613dc44eca5cc42b88d38fed0/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:16.194759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e632b-7f57-42dd-b3c1-c4c95a2cfe44 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.195207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e632b-7f57-42dd-b3c1-c4c95a2cfe44 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.195764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e632b-7f57-42dd-b3c1-c4c95a2cfe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.232437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-fe0e632b-7f57-42dd-b3c1-c4c95a2cfe44 None None] MFA Rules not processed for user `dd650b1613dc44eca5cc42b88d38fed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:16.234741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fe0e632b-7f57-42dd-b3c1-c4c95a2cfe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.298027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.300566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.336239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 993/1981] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:16.347406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e632b-7f57-42dd-b3c1-c4c95a2cfe44 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.349454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fe0e632b-7f57-42dd-b3c1-c4c95a2cfe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.427683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f64f19-5be3-4567-9536-6c27f2739f99 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:16.433879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:16.434694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.435040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.435380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.437856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.443604 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e632b-7f57-42dd-b3c1-c4c95a2cfe44 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:16.451491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:16.452394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.452676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.452953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] PATH_INFO: `/v3/projects/c98413c1705644359db4e3ebe75d8d1f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.454215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c98413c1705644359db4e3ebe75d8d1f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:16.456172 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.503959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:16.504195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:16.522051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c98413c1705644359db4e3ebe75d8d1f', 'request_id': 'req-ab38760a-78e6-48aa-8ad8-c1a398749289'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.531964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c98413c1705644359db4e3ebe75d8d1f', 'request_id': 'req-ab38760a-78e6-48aa-8ad8-c1a398749289'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.532179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:16.534675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.540987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c98413c1705644359db4e3ebe75d8d1f', 'request_id': 'req-ab38760a-78e6-48aa-8ad8-c1a398749289'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.547081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 989/1982] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:16.547474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.548964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.553863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] The token cache is being invalidate because project c98413c1705644359db4e3ebe75d8d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:16.553863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.554474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.557357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 994/1983] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:16 2022] DELETE /identity/v3/projects/c98413c1705644359db4e3ebe75d8d1f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:16.560642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.560982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f2057-ed1e-4ace-bdad-ab6905a957dd service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:16.562431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.618825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:16.619487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.619798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.620080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.622059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.640241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38760a-78e6-48aa-8ad8-c1a398749289 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:16.647079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:16.648431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.648748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.649032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] PATH_INFO: `/v3/users/1356196ef4974fb1a05132b17dbd1f94` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.650829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1356196ef4974fb1a05132b17dbd1f94)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:16.652765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.698356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:16.700097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.710728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 990/1984] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:16.711200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1356196ef4974fb1a05132b17dbd1f94'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.729385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1356196ef4974fb1a05132b17dbd1f94'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.730127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1356196ef4974fb1a05132b17dbd1f94'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.735854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:16.738235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:16.750011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1356196ef4974fb1a05132b17dbd1f94'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.753116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb945d16-7273-4483-be89-5d4511fa3bf1 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:16.759350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:16.760263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.760562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.760844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] PATH_INFO: `/v3/users/dd650b1613dc44eca5cc42b88d38fed0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.761815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1a8ca1ba-0b1e-4feb-bb14-5308834f7901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1356196ef4974fb1a05132b17dbd1f94'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.762216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd650b1613dc44eca5cc42b88d38fed0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:16.764251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:16.769833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 995/1985] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:16 2022] DELETE /identity/v3/users/1356196ef4974fb1a05132b17dbd1f94 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:16.838631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd650b1613dc44eca5cc42b88d38fed0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.839859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd650b1613dc44eca5cc42b88d38fed0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.840627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd650b1613dc44eca5cc42b88d38fed0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.857045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd650b1613dc44eca5cc42b88d38fed0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.866358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd650b1613dc44eca5cc42b88d38fed0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:16.880176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 991/1986] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:16 2022] DELETE /identity/v3/users/dd650b1613dc44eca5cc42b88d38fed0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:16.885359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ce8a-718f-4791-87a2-9b173c23d7c8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:16.885657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ce8a-718f-4791-87a2-9b173c23d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:16.885923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6ce8a-718f-4791-87a2-9b173c23d7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:16.928779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-88d6ce8a-718f-4791-87a2-9b173c23d7c8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:16.933306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-88d6ce8a-718f-4791-87a2-9b173c23d7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.023314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:17.060850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 996/1987] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:17.072841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.075724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.091120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:17.146352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0903249-3446-4370-96e8-a00669b9e31f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.150516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.151900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.152259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.152518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.164644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.173156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.192328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 992/1988] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:17.202532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fff891-314a-4e5f-981b-711c1e96c7ec None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.203572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fff891-314a-4e5f-981b-711c1e96c7ec None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.204046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fff891-314a-4e5f-981b-711c1e96c7ec None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.215806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 997/1989] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:23:17 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:23:17.218476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.223510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.251446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0c02-c379-4ed2-92f4-f0625fdd0684 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.252418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0c02-c379-4ed2-92f4-f0625fdd0684 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.252782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e0c02-c379-4ed2-92f4-f0625fdd0684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.267088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6c99c1-0852-441e-9fa0-eddee8127c24 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.275594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.276515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.284721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.285143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] PATH_INFO: `/v3/projects/8d267d8926d244b8a9fb37781b831df4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.285502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c19e0c02-c379-4ed2-92f4-f0625fdd0684 None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:17.288513 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c19e0c02-c379-4ed2-92f4-f0625fdd0684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.288660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d267d8926d244b8a9fb37781b831df4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.290732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.420861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 998/1990] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:23:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:17.451149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d267d8926d244b8a9fb37781b831df4', 'request_id': 'req-ebb57fa6-f59f-4b9f-955c-11157aefc255'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:17.466605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d267d8926d244b8a9fb37781b831df4', 'request_id': 'req-ebb57fa6-f59f-4b9f-955c-11157aefc255'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:17.483298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d267d8926d244b8a9fb37781b831df4', 'request_id': 'req-ebb57fa6-f59f-4b9f-955c-11157aefc255'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:17.504113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] The token cache is being invalidate because project 8d267d8926d244b8a9fb37781b831df4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:17.504592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:17.505684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:17.511349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 993/1991] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:17 2022] DELETE /identity/v3/projects/8d267d8926d244b8a9fb37781b831df4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:17.513711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dac14-a982-42fd-a200-2ca6fa37b0e3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.514221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dac14-a982-42fd-a200-2ca6fa37b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.514307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dac14-a982-42fd-a200-2ca6fa37b0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.552843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c74dac14-a982-42fd-a200-2ca6fa37b0e3 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:17.554254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c74dac14-a982-42fd-a200-2ca6fa37b0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.593589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.597869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.664062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 999/1992] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:17.666018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557f778-7d99-4118-950c-018bc9cfa88e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.666312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557f778-7d99-4118-950c-018bc9cfa88e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.666681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c557f778-7d99-4118-950c-018bc9cfa88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.697685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c557f778-7d99-4118-950c-018bc9cfa88e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:17.699112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c557f778-7d99-4118-950c-018bc9cfa88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.709196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1000/1993] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:17.710288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557f778-7d99-4118-950c-018bc9cfa88e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.711807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c557f778-7d99-4118-950c-018bc9cfa88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.741243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb57fa6-f59f-4b9f-955c-11157aefc255 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.748963 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.749688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.749975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.750509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.752785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.773229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.774740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.788373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 994/1994] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:17.790979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.806783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c557f778-7d99-4118-950c-018bc9cfa88e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.807002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ecc67-2104-4acf-a1da-3429defe5e89 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.811185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.811862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.812144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.812420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.813430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.814023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.814370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.814664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:17.814801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.815812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:17.816030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.817974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.834783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 995/1995] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:17.839745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.847753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:17.854185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b729f5-eca8-4ac4-a523-e0483beb98c0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:17.860888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33b17c-a442-4b54-b859-ee55d3560680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:17.861926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33b17c-a442-4b54-b859-ee55d3560680 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:17.861926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33b17c-a442-4b54-b859-ee55d3560680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:17.862276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d33b17c-a442-4b54-b859-ee55d3560680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:17.863653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33b17c-a442-4b54-b859-ee55d3560680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.865609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d33b17c-a442-4b54-b859-ee55d3560680 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.873924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:17.919120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:17.920812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 996/1996] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:17 2022] POST /identity/v3/projects => generated 369 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:17.921936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:17.925970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33b17c-a442-4b54-b859-ee55d3560680 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:17.938325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1001/1997] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:18.044764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.046564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:18.051319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d33b17c-a442-4b54-b859-ee55d3560680 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.065197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bb4c8c-53cd-448b-b406-d3feafa5d37f service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.068332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee0fbd-0b19-462f-bff0-9bb2730f93ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.069074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee0fbd-0b19-462f-bff0-9bb2730f93ce admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.069389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee0fbd-0b19-462f-bff0-9bb2730f93ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.069671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee0fbd-0b19-462f-bff0-9bb2730f93ce admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.070377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee0fbd-0b19-462f-bff0-9bb2730f93ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.072622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee0fbd-0b19-462f-bff0-9bb2730f93ce admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.073823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.075162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.076027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.076435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] PATH_INFO: `/v3/projects/6c47df4422604b42b9fd9ec4b99031aa` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.078303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c47df4422604b42b9fd9ec4b99031aa)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.081944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.098686 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 997/1998] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:17 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:23:18.104282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee0fbd-0b19-462f-bff0-9bb2730f93ce admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.119064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee0fbd-0b19-462f-bff0-9bb2730f93ce admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.124683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e7faf-c347-4e4e-9a5d-8763f73a5462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.125969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e7faf-c347-4e4e-9a5d-8763f73a5462 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.126273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e7faf-c347-4e4e-9a5d-8763f73a5462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.126539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59e7faf-c347-4e4e-9a5d-8763f73a5462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.127287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e7faf-c347-4e4e-9a5d-8763f73a5462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.128931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59e7faf-c347-4e4e-9a5d-8763f73a5462 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.141744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 998/1999] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:23:18.143444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c47df4422604b42b9fd9ec4b99031aa', 'request_id': 'req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:18.145710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e7faf-c347-4e4e-9a5d-8763f73a5462 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.153597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c47df4422604b42b9fd9ec4b99031aa', 'request_id': 'req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:18.157672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59e7faf-c347-4e4e-9a5d-8763f73a5462 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.163972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c47df4422604b42b9fd9ec4b99031aa', 'request_id': 'req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:18.178240 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] The token cache is being invalidate because project 6c47df4422604b42b9fd9ec4b99031aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:18.178539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:18.179464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7f4ca1b0-1147-42b7-b7c6-f5631e62fec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:18.182214 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1002/2000] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:18 2022] DELETE /identity/v3/projects/6c47df4422604b42b9fd9ec4b99031aa => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:18.187360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd88af3-4bd8-46c2-819b-d50b442486e4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.187667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd88af3-4bd8-46c2-819b-d50b442486e4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.187946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd88af3-4bd8-46c2-819b-d50b442486e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.212830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7cd88af3-4bd8-46c2-819b-d50b442486e4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:18.215405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7cd88af3-4bd8-46c2-819b-d50b442486e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:18.260086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.262212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.262703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.263517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.264931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.268956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.292579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 999/2001] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:18.297651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.301758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:18.304171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1003/2002] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:18.317407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416aa5f-c319-4f9f-9851-450370bf2915 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.317743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416aa5f-c319-4f9f-9851-450370bf2915 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.318027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416aa5f-c319-4f9f-9851-450370bf2915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.326631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9500569f-a363-450b-aa6a-4e5d5ffe520b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.339236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83795c-459f-443d-acb8-95b42bdc380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.340999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83795c-459f-443d-acb8-95b42bdc380c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.341491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83795c-459f-443d-acb8-95b42bdc380c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.341994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83795c-459f-443d-acb8-95b42bdc380c admin admin] PATH_INFO: `/v3/projects/c2d34586f25847bf902d8bdc494b89da/users/a034fb8b4b214bf29e8d5cc88f62d687/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.352628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4416aa5f-c319-4f9f-9851-450370bf2915 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:18.354273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4416aa5f-c319-4f9f-9851-450370bf2915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:18.366504 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1004/2003] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:18.371878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83795c-459f-443d-acb8-95b42bdc380c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d34586f25847bf902d8bdc494b89da, user_id=a034fb8b4b214bf29e8d5cc88f62d687, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.374062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416aa5f-c319-4f9f-9851-450370bf2915 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.376629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83795c-459f-443d-acb8-95b42bdc380c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.394108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416aa5f-c319-4f9f-9851-450370bf2915 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.404285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1000/2004] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:18 2022] PUT /identity/v3/projects/c2d34586f25847bf902d8bdc494b89da/users/a034fb8b4b214bf29e8d5cc88f62d687/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:18.415232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6969ae6-57ab-43d4-8ee8-49c6ca7c1511 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.415528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6969ae6-57ab-43d4-8ee8-49c6ca7c1511 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.415797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6969ae6-57ab-43d4-8ee8-49c6ca7c1511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.446238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e6969ae6-57ab-43d4-8ee8-49c6ca7c1511 None None] MFA Rules not processed for user `a034fb8b4b214bf29e8d5cc88f62d687`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:18.447815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e6969ae6-57ab-43d4-8ee8-49c6ca7c1511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:18.448977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:18.480610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:18.497551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.498967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.499419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.499703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.500820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:18.501448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.503392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.520053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1005/2005] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:23:18.522005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.528123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1001/2006] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:18.532909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfc56a-f42a-4212-91f7-9e737993cb98 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.538757 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c91b94-7dc6-45d0-b026-31f5ac707be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.539496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91b94-7dc6-45d0-b026-31f5ac707be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.540079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91b94-7dc6-45d0-b026-31f5ac707be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.540238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6969ae6-57ab-43d4-8ee8-49c6ca7c1511 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.540375 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c91b94-7dc6-45d0-b026-31f5ac707be7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.541207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c91b94-7dc6-45d0-b026-31f5ac707be7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.541877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e6969ae6-57ab-43d4-8ee8-49c6ca7c1511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:18.543190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c91b94-7dc6-45d0-b026-31f5ac707be7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.574748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1006/2007] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:18 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:18.579334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c91b94-7dc6-45d0-b026-31f5ac707be7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.610126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6969ae6-57ab-43d4-8ee8-49c6ca7c1511 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.681324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713e4b1-9afb-4170-b98d-e630b337537c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.681999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713e4b1-9afb-4170-b98d-e630b337537c service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.682278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713e4b1-9afb-4170-b98d-e630b337537c service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.682551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713e4b1-9afb-4170-b98d-e630b337537c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.688324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c91b94-7dc6-45d0-b026-31f5ac707be7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.695018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fc6b1-7319-49a5-95a3-806967bcac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.695826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc6b1-7319-49a5-95a3-806967bcac48 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.696281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc6b1-7319-49a5-95a3-806967bcac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.696582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc6b1-7319-49a5-95a3-806967bcac48 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.697644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fc6b1-7319-49a5-95a3-806967bcac48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.700512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66fc6b1-7319-49a5-95a3-806967bcac48 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.735398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1007/2008] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:18 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:18.737892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fc6b1-7319-49a5-95a3-806967bcac48 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.750692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fc6b1-7319-49a5-95a3-806967bcac48 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.756522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8b4ff-b260-464e-9324-b10a65d8112c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.757423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b4ff-b260-464e-9324-b10a65d8112c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.757720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b4ff-b260-464e-9324-b10a65d8112c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.758077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b4ff-b260-464e-9324-b10a65d8112c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.758945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8b4ff-b260-464e-9324-b10a65d8112c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.760730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8b4ff-b260-464e-9324-b10a65d8112c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.762885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713e4b1-9afb-4170-b98d-e630b337537c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.764740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713e4b1-9afb-4170-b98d-e630b337537c service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.772402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1008/2009] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:23:18.776791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8b4ff-b260-464e-9324-b10a65d8112c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.778600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1002/2010] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:18.791003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8b4ff-b260-464e-9324-b10a65d8112c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.797012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1dc7f1-a0a4-4bc4-9bc1-ceb59ac2d7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.797945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1dc7f1-a0a4-4bc4-9bc1-ceb59ac2d7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.798425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1dc7f1-a0a4-4bc4-9bc1-ceb59ac2d7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.798869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1dc7f1-a0a4-4bc4-9bc1-ceb59ac2d7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.799959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1dc7f1-a0a4-4bc4-9bc1-ceb59ac2d7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.802584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1dc7f1-a0a4-4bc4-9bc1-ceb59ac2d7a4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.816027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1009/2011] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:23:18.821709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713e4b1-9afb-4170-b98d-e630b337537c service glance] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:18.840399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713e4b1-9afb-4170-b98d-e630b337537c service glance] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:18.845296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:18.846218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.846501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.846968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] PATH_INFO: `/v3/projects/5fa1f646736b4715a770190347ea0744/users/f705f1fb14ce42668b1f803c74f0aa06/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.879383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa1f646736b4715a770190347ea0744, user_id=f705f1fb14ce42668b1f803c74f0aa06, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:18.882515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:18.898834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1003/2012] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:18 2022] PUT /identity/v3/projects/5fa1f646736b4715a770190347ea0744/users/f705f1fb14ce42668b1f803c74f0aa06/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:18.909661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d411cf-2961-4462-baec-7e1969161d53 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:18.909948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d411cf-2961-4462-baec-7e1969161d53 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:18.910144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d411cf-2961-4462-baec-7e1969161d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:18.959248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e9d411cf-2961-4462-baec-7e1969161d53 None None] MFA Rules not processed for user `f705f1fb14ce42668b1f803c74f0aa06`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:18.960760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e9d411cf-2961-4462-baec-7e1969161d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:19.061155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1010/2013] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:19.174462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:19.176288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:19.200478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:19.292842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546e9f-abe8-40b8-9ff3-0e70cc1cef67 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:19.297346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566c458-8bcc-42d8-a224-e696f2d1d3a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:19.298584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566c458-8bcc-42d8-a224-e696f2d1d3a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:19.298584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566c458-8bcc-42d8-a224-e696f2d1d3a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:19.298839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566c458-8bcc-42d8-a224-e696f2d1d3a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:19.310057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566c458-8bcc-42d8-a224-e696f2d1d3a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:19.311527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566c458-8bcc-42d8-a224-e696f2d1d3a5 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:19.322747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1004/2014] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:19.353912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d411cf-2961-4462-baec-7e1969161d53 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:19.369712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:19.449711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d411cf-2961-4462-baec-7e1969161d53 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:19.455772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:19.457379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:19.457949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:19.458276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] PATH_INFO: `/v3/users/f705f1fb14ce42668b1f803c74f0aa06` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:19.459880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] RBAC: Authorizing `identity:delete_user(user_id=f705f1fb14ce42668b1f803c74f0aa06)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:19.462442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:19.512736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566c458-8bcc-42d8-a224-e696f2d1d3a5 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:19.522125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566c458-8bcc-42d8-a224-e696f2d1d3a5 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:19.527186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:19.529239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:19.529239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:19.529239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:19.529565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f705f1fb14ce42668b1f803c74f0aa06'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:19.530530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:19.550585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f705f1fb14ce42668b1f803c74f0aa06'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:19.552036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f705f1fb14ce42668b1f803c74f0aa06'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:19.563914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f705f1fb14ce42668b1f803c74f0aa06'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:19.575541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-00e49c27-bd6a-4527-99e0-5429f7b432da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f705f1fb14ce42668b1f803c74f0aa06'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:19.584399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1011/2015] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:19 2022] DELETE /identity/v3/users/f705f1fb14ce42668b1f803c74f0aa06 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:19.588905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf40b7-a997-4079-b9fd-00947393535e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:19.589257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf40b7-a997-4079-b9fd-00947393535e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:19.589530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf40b7-a997-4079-b9fd-00947393535e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:19.609684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:19.611139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:19.611667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-76cf40b7-a997-4079-b9fd-00947393535e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:19.612898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-76cf40b7-a997-4079-b9fd-00947393535e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:19.626803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1005/2016] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:19.685593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1012/2017] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:19.697422 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:19.698580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:19.766202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd8b80-f07c-4713-99b8-4f6cea3b2531 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:19.826753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25599e47-d47c-4b8d-9bfe-2ae108112cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:19.827441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599e47-d47c-4b8d-9bfe-2ae108112cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:19.827704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599e47-d47c-4b8d-9bfe-2ae108112cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:19.828397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25599e47-d47c-4b8d-9bfe-2ae108112cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:19.839472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25599e47-d47c-4b8d-9bfe-2ae108112cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:19.841206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25599e47-d47c-4b8d-9bfe-2ae108112cd3 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:19.853410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1006/2018] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:20.921230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf40b7-a997-4079-b9fd-00947393535e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:20.923655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-76cf40b7-a997-4079-b9fd-00947393535e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:20.947835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf40b7-a997-4079-b9fd-00947393535e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:20.955572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:20.956975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:20.957268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:20.957544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] PATH_INFO: `/v3/projects/5fa1f646736b4715a770190347ea0744` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:20.958775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa1f646736b4715a770190347ea0744)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:20.960709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:20.985159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:21.025888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa1f646736b4715a770190347ea0744', 'request_id': 'req-4e30937e-e6fc-4fd7-8900-e544d58bfe24'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:21.035064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa1f646736b4715a770190347ea0744', 'request_id': 'req-4e30937e-e6fc-4fd7-8900-e544d58bfe24'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:21.045268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa1f646736b4715a770190347ea0744', 'request_id': 'req-4e30937e-e6fc-4fd7-8900-e544d58bfe24'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:21.081000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] The token cache is being invalidate because project 5fa1f646736b4715a770190347ea0744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:21.081643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:21.084002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4e30937e-e6fc-4fd7-8900-e544d58bfe24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:21.087515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1013/2019] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:20 2022] DELETE /identity/v3/projects/5fa1f646736b4715a770190347ea0744 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:21.099324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3714bc-2d2a-4f3a-be92-49ed5b270565 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.099865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3714bc-2d2a-4f3a-be92-49ed5b270565 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.099943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3714bc-2d2a-4f3a-be92-49ed5b270565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.136262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-aa3714bc-2d2a-4f3a-be92-49ed5b270565 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:21.136262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aa3714bc-2d2a-4f3a-be92-49ed5b270565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:21.142377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:21.258521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1007/2020] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:21.267801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca67d-c772-44ba-8408-2b3a4857f147 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.268439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca67d-c772-44ba-8408-2b3a4857f147 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.268786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cca67d-c772-44ba-8408-2b3a4857f147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.293838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f3cca67d-c772-44ba-8408-2b3a4857f147 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:21.295061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f3cca67d-c772-44ba-8408-2b3a4857f147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:21.305496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1014/2021] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:21.310355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3714bc-2d2a-4f3a-be92-49ed5b270565 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:21.322157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3714bc-2d2a-4f3a-be92-49ed5b270565 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:21.327099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:21.327866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.328150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.328424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.329160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:21.329498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:21.332283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:21.350644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1008/2022] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:21.351855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca67d-c772-44ba-8408-2b3a4857f147 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:21.361614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cca67d-c772-44ba-8408-2b3a4857f147 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:21.366915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34b8b9-1757-4fa5-aa05-8eb547e02582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:21.367812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b8b9-1757-4fa5-aa05-8eb547e02582 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.368294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b8b9-1757-4fa5-aa05-8eb547e02582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.368802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34b8b9-1757-4fa5-aa05-8eb547e02582 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.369649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34b8b9-1757-4fa5-aa05-8eb547e02582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:21.372094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34b8b9-1757-4fa5-aa05-8eb547e02582 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:21.411071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1015/2023] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:21 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:21.415519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:21.425880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693c9da-8dea-46ea-af15-ba39bb4f2fbd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:21.501471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa451f-9ffc-4df0-a4b4-6ed720bfefc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:21.502361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa451f-9ffc-4df0-a4b4-6ed720bfefc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.502611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa451f-9ffc-4df0-a4b4-6ed720bfefc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.502883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa451f-9ffc-4df0-a4b4-6ed720bfefc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.503587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa451f-9ffc-4df0-a4b4-6ed720bfefc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:21.505897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa451f-9ffc-4df0-a4b4-6ed720bfefc9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:21.532088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1009/2024] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:21 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:21.536188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34b8b9-1757-4fa5-aa05-8eb547e02582 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:21.547976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34b8b9-1757-4fa5-aa05-8eb547e02582 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:21.553727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596d752-5fa7-4d7d-8ba4-2b5533ae50a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:21.554693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596d752-5fa7-4d7d-8ba4-2b5533ae50a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.555035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596d752-5fa7-4d7d-8ba4-2b5533ae50a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.555862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596d752-5fa7-4d7d-8ba4-2b5533ae50a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.556151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596d752-5fa7-4d7d-8ba4-2b5533ae50a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:21.557739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596d752-5fa7-4d7d-8ba4-2b5533ae50a9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:21.569283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1016/2025] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:23:21.573632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa451f-9ffc-4df0-a4b4-6ed720bfefc9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:21.584349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa451f-9ffc-4df0-a4b4-6ed720bfefc9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:21.593835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43548f39-9489-419a-8d53-2146ce968540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:21.594532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43548f39-9489-419a-8d53-2146ce968540 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.594825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43548f39-9489-419a-8d53-2146ce968540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.595092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43548f39-9489-419a-8d53-2146ce968540 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.595838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43548f39-9489-419a-8d53-2146ce968540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:21.597525 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43548f39-9489-419a-8d53-2146ce968540 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:21.609493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1010/2026] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:21.613960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596d752-5fa7-4d7d-8ba4-2b5533ae50a9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:21.627117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596d752-5fa7-4d7d-8ba4-2b5533ae50a9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:21.632901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:21.634275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.634275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.634581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] PATH_INFO: `/v3/projects/bd98cf76b6574417895e5f535e26bb07/users/2a7657c174ec40ba806f3851b6f1435c/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.654303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd98cf76b6574417895e5f535e26bb07, user_id=2a7657c174ec40ba806f3851b6f1435c, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:21.658316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:21.673723 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1017/2027] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:21 2022] PUT /identity/v3/projects/bd98cf76b6574417895e5f535e26bb07/users/2a7657c174ec40ba806f3851b6f1435c/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:21.689292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bc47a-3c75-41a9-a66b-8153f304bafa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.689619 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bc47a-3c75-41a9-a66b-8153f304bafa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.690068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bc47a-3c75-41a9-a66b-8153f304bafa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.724568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-064bc47a-3c75-41a9-a66b-8153f304bafa None None] MFA Rules not processed for user `2a7657c174ec40ba806f3851b6f1435c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:21.725819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-064bc47a-3c75-41a9-a66b-8153f304bafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:21.805948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1011/2028] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:21.822708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:21.825090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:21.893066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77d13-687f-4a3b-84ff-e92344892ec4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:21.897434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0838b60-0677-4c83-8139-41bdcbcc8e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:21.898204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0838b60-0677-4c83-8139-41bdcbcc8e7f service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:21.898500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0838b60-0677-4c83-8139-41bdcbcc8e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:21.898962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0838b60-0677-4c83-8139-41bdcbcc8e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:21.909812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0838b60-0677-4c83-8139-41bdcbcc8e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:21.910948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0838b60-0677-4c83-8139-41bdcbcc8e7f service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:21.922420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1018/2029] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:25.870564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bc47a-3c75-41a9-a66b-8153f304bafa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:25.872115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-064bc47a-3c75-41a9-a66b-8153f304bafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:25.879022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:25.940855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064bc47a-3c75-41a9-a66b-8153f304bafa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:25.945445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:25.946392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:25.946679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:25.946957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] PATH_INFO: `/v3/users/5f34f3fad0a44acf9e28108b05af7b16` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:25.948140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f34f3fad0a44acf9e28108b05af7b16)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:25.950114 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:26.008568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f34f3fad0a44acf9e28108b05af7b16'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.009450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f34f3fad0a44acf9e28108b05af7b16'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.010121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f34f3fad0a44acf9e28108b05af7b16'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.025793 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f34f3fad0a44acf9e28108b05af7b16'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.034691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f34f3fad0a44acf9e28108b05af7b16'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.064079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1012/2030] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:25 2022] DELETE /identity/v3/users/5f34f3fad0a44acf9e28108b05af7b16 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:26.066525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b51e0-e53a-4b42-b993-5afb1800288a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:26.067055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b51e0-e53a-4b42-b993-5afb1800288a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:26.067165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b51e0-e53a-4b42-b993-5afb1800288a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:26.090993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-481b51e0-e53a-4b42-b993-5afb1800288a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:26.091833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-481b51e0-e53a-4b42-b993-5afb1800288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:26.148106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:26.176702 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1019/2031] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:26.183531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:26.185008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:26.246089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a470d-2021-420c-8fa2-3c386dac614a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:26.249844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1c81-5d51-4ae9-9494-3257191e87c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:26.250499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1c81-5d51-4ae9-9494-3257191e87c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:26.250790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1c81-5d51-4ae9-9494-3257191e87c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:26.251072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff1c81-5d51-4ae9-9494-3257191e87c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:26.261265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff1c81-5d51-4ae9-9494-3257191e87c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:26.263604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ff1c81-5d51-4ae9-9494-3257191e87c4 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:26.273820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1013/2032] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:26.626625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b51e0-e53a-4b42-b993-5afb1800288a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:26.629310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-481b51e0-e53a-4b42-b993-5afb1800288a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:26.645810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b51e0-e53a-4b42-b993-5afb1800288a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:26.653049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:26.654206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:26.654467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:26.654928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] PATH_INFO: `/v3/projects/5336d7a5e3bf46fe9dd443a8cf373249` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:26.656151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5336d7a5e3bf46fe9dd443a8cf373249)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:26.658270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:26.743669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5336d7a5e3bf46fe9dd443a8cf373249', 'request_id': 'req-22f52d20-1c6a-4713-afe2-7c2438c87fcc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.785209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5336d7a5e3bf46fe9dd443a8cf373249', 'request_id': 'req-22f52d20-1c6a-4713-afe2-7c2438c87fcc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.799270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5336d7a5e3bf46fe9dd443a8cf373249', 'request_id': 'req-22f52d20-1c6a-4713-afe2-7c2438c87fcc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.812044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] The token cache is being invalidate because project 5336d7a5e3bf46fe9dd443a8cf373249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:26.812471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.813354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:26.818075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1020/2033] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:26 2022] DELETE /identity/v3/projects/5336d7a5e3bf46fe9dd443a8cf373249 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:26.819683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1c81-5d51-4ae9-9494-3257191e87c4 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:26.832278 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff1c81-5d51-4ae9-9494-3257191e87c4 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:26.886937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:26.958625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:26.959538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:26.959823 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:26.960103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] PATH_INFO: `/v3/users/f7ab42130d68466ca814db43f62a1d63` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:26.961894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ab42130d68466ca814db43f62a1d63)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:26.963977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:27.038364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ab42130d68466ca814db43f62a1d63'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.039022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ab42130d68466ca814db43f62a1d63'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.039825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ab42130d68466ca814db43f62a1d63'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.076101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ab42130d68466ca814db43f62a1d63'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.105025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ab42130d68466ca814db43f62a1d63'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.136997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1014/2034] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:26 2022] DELETE /identity/v3/users/f7ab42130d68466ca814db43f62a1d63 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:27.751806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:27.751806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:27.770601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f52d20-1c6a-4713-afe2-7c2438c87fcc admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:27.827712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:27.828578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:27.828868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:27.829146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] PATH_INFO: `/v3/projects/e3dd833359504229bc9b4e29a421a440` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:27.830360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3dd833359504229bc9b4e29a421a440)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:27.832258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:27.879346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3dd833359504229bc9b4e29a421a440', 'request_id': 'req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.887529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3dd833359504229bc9b4e29a421a440', 'request_id': 'req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.894231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3dd833359504229bc9b4e29a421a440', 'request_id': 'req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.901432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:27.905846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] The token cache is being invalidate because project e3dd833359504229bc9b4e29a421a440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:27.906251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.907293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:27.909134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1021/2035] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:27 2022] DELETE /identity/v3/projects/e3dd833359504229bc9b4e29a421a440 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:27.913363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:27.914942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:27.928207 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff32b84-94f0-4409-9ac9-71ceeefcc05e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:27.988340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:27.989203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:27.989492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:27.989816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] PATH_INFO: `/v3/users/3e7d6b08b9384a79b836716527adec08` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:27.991031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e7d6b08b9384a79b836716527adec08)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:27.993014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:28.039118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e7d6b08b9384a79b836716527adec08'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.040016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e7d6b08b9384a79b836716527adec08'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.040640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e7d6b08b9384a79b836716527adec08'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.041835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:28.055372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e7d6b08b9384a79b836716527adec08'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.063156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f016b2d7-b27e-41cc-a8fe-d501d789bd5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e7d6b08b9384a79b836716527adec08'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.076724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1015/2036] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:27 2022] DELETE /identity/v3/users/3e7d6b08b9384a79b836716527adec08 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:28.353689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:28.355216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:28.375158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc91bc8-c8be-4c6d-adb3-9118c27ebd2a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:28.432623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:28.433539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:28.433824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:28.434101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] PATH_INFO: `/v3/projects/df85f6946e424a6aa1dd55bf029bfe3c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:28.435295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df85f6946e424a6aa1dd55bf029bfe3c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:28.437223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:28.504174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df85f6946e424a6aa1dd55bf029bfe3c', 'request_id': 'req-99f34d91-f325-4ba2-acd1-2e7661bd11e5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.513495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df85f6946e424a6aa1dd55bf029bfe3c', 'request_id': 'req-99f34d91-f325-4ba2-acd1-2e7661bd11e5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.521698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df85f6946e424a6aa1dd55bf029bfe3c', 'request_id': 'req-99f34d91-f325-4ba2-acd1-2e7661bd11e5'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.539147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] The token cache is being invalidate because project df85f6946e424a6aa1dd55bf029bfe3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:28.539679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.540437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-99f34d91-f325-4ba2-acd1-2e7661bd11e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:28.542055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1022/2037] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:28 2022] DELETE /identity/v3/projects/df85f6946e424a6aa1dd55bf029bfe3c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:28.547746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6fd9d-2c49-44fa-a0ea-131b8d431a51 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:28.548030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6fd9d-2c49-44fa-a0ea-131b8d431a51 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:28.548307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6fd9d-2c49-44fa-a0ea-131b8d431a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:28.567250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-03e6fd9d-2c49-44fa-a0ea-131b8d431a51 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:28.568449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-03e6fd9d-2c49-44fa-a0ea-131b8d431a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:28.643681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1016/2038] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:28.652994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb14bf-d3df-4545-8fd9-b91fd3a1c3ca None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:28.653311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb14bf-d3df-4545-8fd9-b91fd3a1c3ca None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:28.653588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb14bf-d3df-4545-8fd9-b91fd3a1c3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:28.676223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-adeb14bf-d3df-4545-8fd9-b91fd3a1c3ca None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:28.676659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-adeb14bf-d3df-4545-8fd9-b91fd3a1c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:28.686602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1023/2039] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:28.690955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6fd9d-2c49-44fa-a0ea-131b8d431a51 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:28.701808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6fd9d-2c49-44fa-a0ea-131b8d431a51 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:28.708564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:28.709759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:28.710198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:28.710703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:28.711803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:28.712202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:28.715441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:28.732582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1017/2040] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:28.737191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb14bf-d3df-4545-8fd9-b91fd3a1c3ca None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:28.747536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb14bf-d3df-4545-8fd9-b91fd3a1c3ca None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:28.753083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236b9e3-38f2-49e8-97bc-e42fc0c93dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:28.753968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236b9e3-38f2-49e8-97bc-e42fc0c93dd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:28.754260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236b9e3-38f2-49e8-97bc-e42fc0c93dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:28.754543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236b9e3-38f2-49e8-97bc-e42fc0c93dd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:28.755605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236b9e3-38f2-49e8-97bc-e42fc0c93dd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:28.757575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236b9e3-38f2-49e8-97bc-e42fc0c93dd7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:28.797006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1024/2041] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:28 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:28.802472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:28.814792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1242c8-2f10-47bd-b526-281603f07be3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:28.892578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617b206-f787-4124-b1c4-642453bd390b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:28.892578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617b206-f787-4124-b1c4-642453bd390b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:28.892578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617b206-f787-4124-b1c4-642453bd390b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:28.892578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617b206-f787-4124-b1c4-642453bd390b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:28.893070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617b206-f787-4124-b1c4-642453bd390b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:28.893070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617b206-f787-4124-b1c4-642453bd390b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:28.912667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1018/2042] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:28 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:23:28.917422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236b9e3-38f2-49e8-97bc-e42fc0c93dd7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:28.927731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236b9e3-38f2-49e8-97bc-e42fc0c93dd7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:28.932930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257ec83-78f7-4c7e-b59e-a7b2bc92aa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:28.933743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257ec83-78f7-4c7e-b59e-a7b2bc92aa34 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:28.934193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257ec83-78f7-4c7e-b59e-a7b2bc92aa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:28.934489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257ec83-78f7-4c7e-b59e-a7b2bc92aa34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:28.935326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257ec83-78f7-4c7e-b59e-a7b2bc92aa34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:28.937071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257ec83-78f7-4c7e-b59e-a7b2bc92aa34 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:28.949300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1025/2043] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:28.953913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617b206-f787-4124-b1c4-642453bd390b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:28.965641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617b206-f787-4124-b1c4-642453bd390b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:28.971845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1172f39-b4a1-4e42-8fdf-d7fb6ca5c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:28.972518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1172f39-b4a1-4e42-8fdf-d7fb6ca5c384 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:28.972796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1172f39-b4a1-4e42-8fdf-d7fb6ca5c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:28.973067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1172f39-b4a1-4e42-8fdf-d7fb6ca5c384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:28.973836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1172f39-b4a1-4e42-8fdf-d7fb6ca5c384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:28.975714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1172f39-b4a1-4e42-8fdf-d7fb6ca5c384 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:28.987273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1019/2044] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:28.993483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257ec83-78f7-4c7e-b59e-a7b2bc92aa34 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:29.007733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257ec83-78f7-4c7e-b59e-a7b2bc92aa34 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:29.013075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:29.014080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:29.014365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:29.014672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] PATH_INFO: `/v3/projects/1bc238aba75b405a9e073025c1e1fb85/users/c8013b2b7a954f2d8fdd7e7d51ccf6f1/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:29.032374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc238aba75b405a9e073025c1e1fb85, user_id=c8013b2b7a954f2d8fdd7e7d51ccf6f1, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:29.034997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:29.054184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1026/2045] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:28 2022] PUT /identity/v3/projects/1bc238aba75b405a9e073025c1e1fb85/users/c8013b2b7a954f2d8fdd7e7d51ccf6f1/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:29.066847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29148b87-5e7d-4966-8b59-7a56375d6769 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:29.067208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29148b87-5e7d-4966-8b59-7a56375d6769 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:29.067611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29148b87-5e7d-4966-8b59-7a56375d6769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:29.098342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-29148b87-5e7d-4966-8b59-7a56375d6769 None None] MFA Rules not processed for user `c8013b2b7a954f2d8fdd7e7d51ccf6f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:29.099541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-29148b87-5e7d-4966-8b59-7a56375d6769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:29.126830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:29.181878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1020/2046] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:29.194515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:29.197032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:29.229961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:29.258912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21e7da2-2c82-49a1-8895-4f1f51234f16 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:29.262898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10cb64-0b13-4b2d-808e-3622d6b48ab7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:29.263552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cb64-0b13-4b2d-808e-3622d6b48ab7 service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:29.263835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cb64-0b13-4b2d-808e-3622d6b48ab7 service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:29.264093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10cb64-0b13-4b2d-808e-3622d6b48ab7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:29.272784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10cb64-0b13-4b2d-808e-3622d6b48ab7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:29.274328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10cb64-0b13-4b2d-808e-3622d6b48ab7 service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:29.283235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1027/2047] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:39.177441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=18 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:39.279643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=14 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:46.765707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29148b87-5e7d-4966-8b59-7a56375d6769 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:46.767191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-29148b87-5e7d-4966-8b59-7a56375d6769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:46.841233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29148b87-5e7d-4966-8b59-7a56375d6769 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:46.845831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:46.846779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:46.847068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:46.847346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] PATH_INFO: `/v3/users/573b7fe0ff054b299a5bfbb7d8eee6ee` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:46.848597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=573b7fe0ff054b299a5bfbb7d8eee6ee)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:46.850565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:46.908865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '573b7fe0ff054b299a5bfbb7d8eee6ee'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:46.909610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '573b7fe0ff054b299a5bfbb7d8eee6ee'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:46.910320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '573b7fe0ff054b299a5bfbb7d8eee6ee'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:46.933019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '573b7fe0ff054b299a5bfbb7d8eee6ee'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:46.948012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '573b7fe0ff054b299a5bfbb7d8eee6ee'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:46.971706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1021/2048] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:46 2022] DELETE /identity/v3/users/573b7fe0ff054b299a5bfbb7d8eee6ee => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:46.981667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672dd6-a368-45cc-88a1-c5deb4e7ff4d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:46.982606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672dd6-a368-45cc-88a1-c5deb4e7ff4d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:46.982916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb672dd6-a368-45cc-88a1-c5deb4e7ff4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:47.014143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-eb672dd6-a368-45cc-88a1-c5deb4e7ff4d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:47.016319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-eb672dd6-a368-45cc-88a1-c5deb4e7ff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:47.139414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1028/2049] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:47.154753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:47.157085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:47.257242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1671c5-a17a-4bdd-b4ba-a0434c9ed023 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:47.263359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0d11-f08c-4a91-ae25-39454b031f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:47.264497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0d11-f08c-4a91-ae25-39454b031f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:47.264979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0d11-f08c-4a91-ae25-39454b031f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:47.265503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a0d11-f08c-4a91-ae25-39454b031f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:47.281625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0d11-f08c-4a91-ae25-39454b031f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:47.284211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a0d11-f08c-4a91-ae25-39454b031f99 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:47.301789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1022/2050] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:47.731035 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672dd6-a368-45cc-88a1-c5deb4e7ff4d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:47.732823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-eb672dd6-a368-45cc-88a1-c5deb4e7ff4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:47.753299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb672dd6-a368-45cc-88a1-c5deb4e7ff4d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:47.759782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:47.761460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:47.762136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:47.762594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] PATH_INFO: `/v3/projects/acaf852fa3b041aeb42927629894b678` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:47.764915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acaf852fa3b041aeb42927629894b678)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:47.771415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:47.832404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acaf852fa3b041aeb42927629894b678', 'request_id': 'req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:47.842557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acaf852fa3b041aeb42927629894b678', 'request_id': 'req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:47.849264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acaf852fa3b041aeb42927629894b678', 'request_id': 'req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:47.859761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] The token cache is being invalidate because project acaf852fa3b041aeb42927629894b678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:47.860042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:47.860663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:47.862244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1029/2051] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:47 2022] DELETE /identity/v3/projects/acaf852fa3b041aeb42927629894b678 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:47.867395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0d11-f08c-4a91-ae25-39454b031f99 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:47.877837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a0d11-f08c-4a91-ae25-39454b031f99 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:47.896570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:47.940347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:47.941336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:47.941680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:47.942023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] PATH_INFO: `/v3/users/e8b1819ec2dd4306aa0584f6de14074a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:47.943469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b1819ec2dd4306aa0584f6de14074a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:47.945532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:47.998024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b1819ec2dd4306aa0584f6de14074a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:47.998646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b1819ec2dd4306aa0584f6de14074a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:47.999249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b1819ec2dd4306aa0584f6de14074a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:48.011645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b1819ec2dd4306aa0584f6de14074a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:48.020559 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f6f67c3-40c1-4a17-ac85-3d6141e9aec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b1819ec2dd4306aa0584f6de14074a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:48.035556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1023/2052] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:47 2022] DELETE /identity/v3/users/e8b1819ec2dd4306aa0584f6de14074a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:48.330598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:48.332364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:48.346427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ea5c7-9f98-44e3-a2c6-c7cec060df90 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:48.420122 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:48.420997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.421310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.421583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] PATH_INFO: `/v3/projects/7a7f85339b3d4dd9aa8b4a5274d5bd3c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.422782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7f85339b3d4dd9aa8b4a5274d5bd3c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:48.424676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:48.459082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:48.479088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7f85339b3d4dd9aa8b4a5274d5bd3c', 'request_id': 'req-199410e6-c85b-4136-8f76-17011f803519'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:48.489004 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7f85339b3d4dd9aa8b4a5274d5bd3c', 'request_id': 'req-199410e6-c85b-4136-8f76-17011f803519'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:48.496867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7f85339b3d4dd9aa8b4a5274d5bd3c', 'request_id': 'req-199410e6-c85b-4136-8f76-17011f803519'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:48.510140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] The token cache is being invalidate because project 7a7f85339b3d4dd9aa8b4a5274d5bd3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:48.510518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:48.511357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199410e6-c85b-4136-8f76-17011f803519 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:48.512970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1030/2053] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:48 2022] DELETE /identity/v3/projects/7a7f85339b3d4dd9aa8b4a5274d5bd3c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:48.518042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f973731-67af-4dbf-ac86-21ad9a19ac35 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.518271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f973731-67af-4dbf-ac86-21ad9a19ac35 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.518586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f973731-67af-4dbf-ac86-21ad9a19ac35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.537114 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4f973731-67af-4dbf-ac86-21ad9a19ac35 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:48.538358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f973731-67af-4dbf-ac86-21ad9a19ac35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:48.611670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1024/2054] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:48.619864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc787d-b61c-4b26-b690-51856cbef627 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.620149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc787d-b61c-4b26-b690-51856cbef627 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.620797 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc787d-b61c-4b26-b690-51856cbef627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.642485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4dcc787d-b61c-4b26-b690-51856cbef627 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:48.643694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4dcc787d-b61c-4b26-b690-51856cbef627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:48.652408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1031/2055] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:48.656978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f973731-67af-4dbf-ac86-21ad9a19ac35 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:48.667427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f973731-67af-4dbf-ac86-21ad9a19ac35 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:48.672884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:48.673744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.674032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.674341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.675076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:48.675379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:48.677005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:48.690101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1025/2056] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:48.694565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc787d-b61c-4b26-b690-51856cbef627 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:48.706405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc787d-b61c-4b26-b690-51856cbef627 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:48.711365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c049d89-7ad1-4415-abab-d8915faf01d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:48.712037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c049d89-7ad1-4415-abab-d8915faf01d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.712312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c049d89-7ad1-4415-abab-d8915faf01d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.712584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c049d89-7ad1-4415-abab-d8915faf01d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.713305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c049d89-7ad1-4415-abab-d8915faf01d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:48.715380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c049d89-7ad1-4415-abab-d8915faf01d2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:48.750508 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1032/2057] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:48 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:48.751804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:48.762622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaa2ef-2948-4dba-aa9f-ad5e3a946d33 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:48.802718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:48.847138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc290-5c15-4a2d-8cc3-d6f07c5fe963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:48.847817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc290-5c15-4a2d-8cc3-d6f07c5fe963 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.848268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc290-5c15-4a2d-8cc3-d6f07c5fe963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.848550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918cc290-5c15-4a2d-8cc3-d6f07c5fe963 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.849252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cc290-5c15-4a2d-8cc3-d6f07c5fe963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:48.851179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918cc290-5c15-4a2d-8cc3-d6f07c5fe963 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:48.876309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1026/2058] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:48 2022] POST /identity/v3/users => generated 291 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:48.880263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c049d89-7ad1-4415-abab-d8915faf01d2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:48.891908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c049d89-7ad1-4415-abab-d8915faf01d2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:48.897166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef564a84-5086-4d55-a835-4c1a95b7ccf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:48.897878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564a84-5086-4d55-a835-4c1a95b7ccf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.898159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564a84-5086-4d55-a835-4c1a95b7ccf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.898434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef564a84-5086-4d55-a835-4c1a95b7ccf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.899468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef564a84-5086-4d55-a835-4c1a95b7ccf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:48.901093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef564a84-5086-4d55-a835-4c1a95b7ccf9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:48.912349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1033/2059] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:48.917286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc290-5c15-4a2d-8cc3-d6f07c5fe963 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:48.930763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918cc290-5c15-4a2d-8cc3-d6f07c5fe963 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:48.938894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918cab5-5fb0-4823-8e4a-eae7dd8f6ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:48.939576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918cab5-5fb0-4823-8e4a-eae7dd8f6ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.939863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918cab5-5fb0-4823-8e4a-eae7dd8f6ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.940131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918cab5-5fb0-4823-8e4a-eae7dd8f6ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.940878 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918cab5-5fb0-4823-8e4a-eae7dd8f6ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:48.942560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918cab5-5fb0-4823-8e4a-eae7dd8f6ec4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:48.954017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1027/2060] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:48.958545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef564a84-5086-4d55-a835-4c1a95b7ccf9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:48.969685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef564a84-5086-4d55-a835-4c1a95b7ccf9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:48.975650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1d65f-7625-4b13-95c9-274a249fc0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:48.976624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d65f-7625-4b13-95c9-274a249fc0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:48.977024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d65f-7625-4b13-95c9-274a249fc0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:48.977530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d65f-7625-4b13-95c9-274a249fc0c1 admin admin] PATH_INFO: `/v3/projects/847cf68de2f043149670cf5d3fe2fe48/users/caeaecc6a85a40d2aca61666aed1c42e/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:48.994469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1d65f-7625-4b13-95c9-274a249fc0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=847cf68de2f043149670cf5d3fe2fe48, user_id=caeaecc6a85a40d2aca61666aed1c42e, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:48.997146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1d65f-7625-4b13-95c9-274a249fc0c1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:49.012846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1034/2061] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:48 2022] PUT /identity/v3/projects/847cf68de2f043149670cf5d3fe2fe48/users/caeaecc6a85a40d2aca61666aed1c42e/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:49.025159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55478f-6b8e-429c-8cc7-59d7e3ac7f49 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.025487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55478f-6b8e-429c-8cc7-59d7e3ac7f49 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.025770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55478f-6b8e-429c-8cc7-59d7e3ac7f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.055925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5f55478f-6b8e-429c-8cc7-59d7e3ac7f49 None None] MFA Rules not processed for user `caeaecc6a85a40d2aca61666aed1c42e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:49.057415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f55478f-6b8e-429c-8cc7-59d7e3ac7f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:49.137534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1028/2062] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:49.142177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1d65f-7625-4b13-95c9-274a249fc0c1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:49.225562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1d65f-7625-4b13-95c9-274a249fc0c1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:49.232658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87040790-0326-47b3-8140-f12ca04799f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:49.234374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040790-0326-47b3-8140-f12ca04799f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.234665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040790-0326-47b3-8140-f12ca04799f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.234943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87040790-0326-47b3-8140-f12ca04799f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.236291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87040790-0326-47b3-8140-f12ca04799f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:49.241251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87040790-0326-47b3-8140-f12ca04799f3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:49.281445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1035/2063] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:49 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:49.287659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55478f-6b8e-429c-8cc7-59d7e3ac7f49 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:49.303997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55478f-6b8e-429c-8cc7-59d7e3ac7f49 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:49.381885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fc0bf-f0d1-4dad-9aea-40fd65b70d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:49.382446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fc0bf-f0d1-4dad-9aea-40fd65b70d8a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.382720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fc0bf-f0d1-4dad-9aea-40fd65b70d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.383063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fc0bf-f0d1-4dad-9aea-40fd65b70d8a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.383782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fc0bf-f0d1-4dad-9aea-40fd65b70d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:49.385715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fc0bf-f0d1-4dad-9aea-40fd65b70d8a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:49.408674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1029/2064] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:49 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:49.413202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87040790-0326-47b3-8140-f12ca04799f3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:49.425144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87040790-0326-47b3-8140-f12ca04799f3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:49.431208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c94d9a-7985-4582-9fbc-9650f1e7000f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:49.431887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c94d9a-7985-4582-9fbc-9650f1e7000f admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.432183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c94d9a-7985-4582-9fbc-9650f1e7000f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.432482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c94d9a-7985-4582-9fbc-9650f1e7000f admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.433223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c94d9a-7985-4582-9fbc-9650f1e7000f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:49.434821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c94d9a-7985-4582-9fbc-9650f1e7000f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:49.451284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1036/2065] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:49.455836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fc0bf-f0d1-4dad-9aea-40fd65b70d8a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:49.468812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fc0bf-f0d1-4dad-9aea-40fd65b70d8a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:49.474728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c7a3-5763-49b7-b38b-76bcba4bbb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:49.475639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c7a3-5763-49b7-b38b-76bcba4bbb2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.475916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c7a3-5763-49b7-b38b-76bcba4bbb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.476188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c7a3-5763-49b7-b38b-76bcba4bbb2e admin admin] PATH_INFO: `/v3/projects/4009f64a4afa4fff8a34314674fe36e0/users/43ddf617ba5240d9b7b6ae24c75c7b8a/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.500395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c7a3-5763-49b7-b38b-76bcba4bbb2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4009f64a4afa4fff8a34314674fe36e0, user_id=43ddf617ba5240d9b7b6ae24c75c7b8a, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:49.503224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c7a3-5763-49b7-b38b-76bcba4bbb2e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:49.521056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1030/2066] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:49 2022] PUT /identity/v3/projects/4009f64a4afa4fff8a34314674fe36e0/users/43ddf617ba5240d9b7b6ae24c75c7b8a/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:49.525892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c94d9a-7985-4582-9fbc-9650f1e7000f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:49.539278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c94d9a-7985-4582-9fbc-9650f1e7000f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:49.577409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:49.630637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9159c-50f5-4f2e-ab79-1e6e0957d65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:49.631307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9159c-50f5-4f2e-ab79-1e6e0957d65a admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.631589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9159c-50f5-4f2e-ab79-1e6e0957d65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.631870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9159c-50f5-4f2e-ab79-1e6e0957d65a admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.632697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9159c-50f5-4f2e-ab79-1e6e0957d65a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:49.634440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9159c-50f5-4f2e-ab79-1e6e0957d65a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:49.645926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1037/2067] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:49.650510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c7a3-5763-49b7-b38b-76bcba4bbb2e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:49.656868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:49.664248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c7a3-5763-49b7-b38b-76bcba4bbb2e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:49.671905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:49.672870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.673301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.673442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] PATH_INFO: `/v3/projects/4009f64a4afa4fff8a34314674fe36e0/users/43ddf617ba5240d9b7b6ae24c75c7b8a/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.675756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4009f64a4afa4fff8a34314674fe36e0, user_id=43ddf617ba5240d9b7b6ae24c75c7b8a, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:49.678469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:49.691023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1031/2068] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:49 2022] PUT /identity/v3/projects/4009f64a4afa4fff8a34314674fe36e0/users/43ddf617ba5240d9b7b6ae24c75c7b8a/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:49.702845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8b32a-6e22-47ca-9dc5-85f6ff1abc6a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.703140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8b32a-6e22-47ca-9dc5-85f6ff1abc6a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.703416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8b32a-6e22-47ca-9dc5-85f6ff1abc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.734271 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cff8b32a-6e22-47ca-9dc5-85f6ff1abc6a None None] MFA Rules not processed for user `43ddf617ba5240d9b7b6ae24c75c7b8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:49.735695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cff8b32a-6e22-47ca-9dc5-85f6ff1abc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:49.811709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1038/2069] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:49.825255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:49.827995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:49.892463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a189458-2994-468a-88f4-b1a2112a2165 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:49.897894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8307e42-144e-4cca-9c7c-7fb8078849a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:49.900078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8307e42-144e-4cca-9c7c-7fb8078849a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:49.900526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8307e42-144e-4cca-9c7c-7fb8078849a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:49.900999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8307e42-144e-4cca-9c7c-7fb8078849a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:49.915181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8307e42-144e-4cca-9c7c-7fb8078849a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:49.916679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8307e42-144e-4cca-9c7c-7fb8078849a1 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:49.927769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1032/2070] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:50.028587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8b32a-6e22-47ca-9dc5-85f6ff1abc6a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:50.038552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff8b32a-6e22-47ca-9dc5-85f6ff1abc6a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:50.043169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2ed9d-396a-49e8-b726-c5dbdd9d6860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:50.043817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2ed9d-396a-49e8-b726-c5dbdd9d6860 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:50.044108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2ed9d-396a-49e8-b726-c5dbdd9d6860 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:50.044377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2ed9d-396a-49e8-b726-c5dbdd9d6860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:50.103510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2ed9d-396a-49e8-b726-c5dbdd9d6860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:50.104959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2ed9d-396a-49e8-b726-c5dbdd9d6860 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:50.115120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1039/2071] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:52.735172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8307e42-144e-4cca-9c7c-7fb8078849a1 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:52.800191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8307e42-144e-4cca-9c7c-7fb8078849a1 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:52.805040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:52.805782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:52.806061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:52.806340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] PATH_INFO: `/v3/users/caeaecc6a85a40d2aca61666aed1c42e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:52.807576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caeaecc6a85a40d2aca61666aed1c42e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:52.809563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:52.859633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caeaecc6a85a40d2aca61666aed1c42e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:52.860271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caeaecc6a85a40d2aca61666aed1c42e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:52.860866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caeaecc6a85a40d2aca61666aed1c42e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:52.883119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caeaecc6a85a40d2aca61666aed1c42e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:52.891881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caeaecc6a85a40d2aca61666aed1c42e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:52.905110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1033/2072] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:52 2022] DELETE /identity/v3/users/caeaecc6a85a40d2aca61666aed1c42e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:52.910031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83167e-f962-43b4-99b5-98536247f547 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:52.910493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83167e-f962-43b4-99b5-98536247f547 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:52.910599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83167e-f962-43b4-99b5-98536247f547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:52.931472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ed83167e-f962-43b4-99b5-98536247f547 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:52.932717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ed83167e-f962-43b4-99b5-98536247f547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:52.980921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:53.003433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1040/2073] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:53.014521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:53.016395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:53.032731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:53.081104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2197701-1010-49bf-8ef8-d82e6283cb32 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:53.085204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc08f6-16dc-4f1a-a88e-c27f9e4c5ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:53.085869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc08f6-16dc-4f1a-a88e-c27f9e4c5ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:53.086395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc08f6-16dc-4f1a-a88e-c27f9e4c5ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:53.086804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc08f6-16dc-4f1a-a88e-c27f9e4c5ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:53.097300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc08f6-16dc-4f1a-a88e-c27f9e4c5ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:53.098750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc08f6-16dc-4f1a-a88e-c27f9e4c5ca2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:53.109560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1034/2074] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:53.422879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83167e-f962-43b4-99b5-98536247f547 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:53.424404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ed83167e-f962-43b4-99b5-98536247f547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:53.436816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83167e-f962-43b4-99b5-98536247f547 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:53.442201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:53.443210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:53.443493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:53.443769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] PATH_INFO: `/v3/projects/847cf68de2f043149670cf5d3fe2fe48` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:53.445000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=847cf68de2f043149670cf5d3fe2fe48)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:53.447724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:53.523000 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '847cf68de2f043149670cf5d3fe2fe48', 'request_id': 'req-43c942d3-f64f-4658-90a5-aa67521ce0a7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.532346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '847cf68de2f043149670cf5d3fe2fe48', 'request_id': 'req-43c942d3-f64f-4658-90a5-aa67521ce0a7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.539031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '847cf68de2f043149670cf5d3fe2fe48', 'request_id': 'req-43c942d3-f64f-4658-90a5-aa67521ce0a7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.555185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] The token cache is being invalidate because project 847cf68de2f043149670cf5d3fe2fe48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:53.555499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.556448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.558041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1041/2075] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:53 2022] DELETE /identity/v3/projects/847cf68de2f043149670cf5d3fe2fe48 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:53.562130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc08f6-16dc-4f1a-a88e-c27f9e4c5ca2 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:53.572668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc08f6-16dc-4f1a-a88e-c27f9e4c5ca2 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:53.641861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:53.642743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:53.643050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:53.643324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] PATH_INFO: `/v3/users/43ddf617ba5240d9b7b6ae24c75c7b8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:53.644585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ddf617ba5240d9b7b6ae24c75c7b8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:53.646585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:53.713860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ddf617ba5240d9b7b6ae24c75c7b8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.714771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ddf617ba5240d9b7b6ae24c75c7b8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.715485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ddf617ba5240d9b7b6ae24c75c7b8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.729636 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ddf617ba5240d9b7b6ae24c75c7b8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.740578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8fd4c12f-56c0-40d4-9af4-852d030136bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ddf617ba5240d9b7b6ae24c75c7b8a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:53.756557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1035/2076] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:53 2022] DELETE /identity/v3/users/43ddf617ba5240d9b7b6ae24c75c7b8a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:54.052769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.055085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:54.070787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c942d3-f64f-4658-90a5-aa67521ce0a7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:54.144823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:54.145747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.146106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.146531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] PATH_INFO: `/v3/projects/4009f64a4afa4fff8a34314674fe36e0` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.148022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4009f64a4afa4fff8a34314674fe36e0)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:54.150280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:54.201147 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4009f64a4afa4fff8a34314674fe36e0', 'request_id': 'req-214ef290-64ef-427f-b97c-d26943b741cf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:54.209846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4009f64a4afa4fff8a34314674fe36e0', 'request_id': 'req-214ef290-64ef-427f-b97c-d26943b741cf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:54.216883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4009f64a4afa4fff8a34314674fe36e0', 'request_id': 'req-214ef290-64ef-427f-b97c-d26943b741cf'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:54.223275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:54.229303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] The token cache is being invalidate because project 4009f64a4afa4fff8a34314674fe36e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:54.229686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:54.230607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-214ef290-64ef-427f-b97c-d26943b741cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:54.233318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1042/2077] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:54 2022] DELETE /identity/v3/projects/4009f64a4afa4fff8a34314674fe36e0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:54.240222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d722f-5d35-4c9e-b5af-0ae7d842879c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.240542 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d722f-5d35-4c9e-b5af-0ae7d842879c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.240794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2d722f-5d35-4c9e-b5af-0ae7d842879c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.261588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1f2d722f-5d35-4c9e-b5af-0ae7d842879c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:54.262807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f2d722f-5d35-4c9e-b5af-0ae7d842879c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:54.293257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:54.332784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1036/2078] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:54.341668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2391d3-5b9d-4014-89ad-7bc76d2e8f8f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.341977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2391d3-5b9d-4014-89ad-7bc76d2e8f8f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.342262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2391d3-5b9d-4014-89ad-7bc76d2e8f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.362479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8e2391d3-5b9d-4014-89ad-7bc76d2e8f8f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:54.363691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8e2391d3-5b9d-4014-89ad-7bc76d2e8f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:54.372545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1043/2079] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:54.377560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d722f-5d35-4c9e-b5af-0ae7d842879c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.390119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2d722f-5d35-4c9e-b5af-0ae7d842879c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:54.396070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:54.396743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.397022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.397337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.398069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:54.398377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:54.400092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:54.412328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1037/2080] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:54.417132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2391d3-5b9d-4014-89ad-7bc76d2e8f8f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.428356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2391d3-5b9d-4014-89ad-7bc76d2e8f8f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:54.434274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf8e6d-f9b1-4abf-aa32-f1735297bfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:54.434927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf8e6d-f9b1-4abf-aa32-f1735297bfbf admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.435208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf8e6d-f9b1-4abf-aa32-f1735297bfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.435480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf8e6d-f9b1-4abf-aa32-f1735297bfbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.436169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf8e6d-f9b1-4abf-aa32-f1735297bfbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:54.438085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf8e6d-f9b1-4abf-aa32-f1735297bfbf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:54.475921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1044/2081] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:54 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:54.480437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.491262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbce8df-8f19-4078-87c0-ef3028c68a55 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:54.556136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6160f-3458-46fb-91c1-d229ee019343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:54.556846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6160f-3458-46fb-91c1-d229ee019343 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.557147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6160f-3458-46fb-91c1-d229ee019343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.557452 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6160f-3458-46fb-91c1-d229ee019343 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.558198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6160f-3458-46fb-91c1-d229ee019343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:54.560141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6160f-3458-46fb-91c1-d229ee019343 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:54.584848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1038/2082] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:54 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:54.588974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf8e6d-f9b1-4abf-aa32-f1735297bfbf admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.600396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf8e6d-f9b1-4abf-aa32-f1735297bfbf admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:54.605491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36d61f-a694-4d41-885b-ffd7d658bbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:54.606283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36d61f-a694-4d41-885b-ffd7d658bbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.606906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36d61f-a694-4d41-885b-ffd7d658bbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.607195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36d61f-a694-4d41-885b-ffd7d658bbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.607926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36d61f-a694-4d41-885b-ffd7d658bbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:54.609789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df36d61f-a694-4d41-885b-ffd7d658bbc9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:54.625750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1045/2083] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:54.627002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6160f-3458-46fb-91c1-d229ee019343 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.637743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6160f-3458-46fb-91c1-d229ee019343 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:54.642528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999644f-8d0e-45c5-a1bf-ed6af1be6257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:54.643206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999644f-8d0e-45c5-a1bf-ed6af1be6257 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.643485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999644f-8d0e-45c5-a1bf-ed6af1be6257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.643757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999644f-8d0e-45c5-a1bf-ed6af1be6257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.644492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999644f-8d0e-45c5-a1bf-ed6af1be6257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:54.646125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9999644f-8d0e-45c5-a1bf-ed6af1be6257 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:54.657469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1039/2084] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:23:54.661688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36d61f-a694-4d41-885b-ffd7d658bbc9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.673037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36d61f-a694-4d41-885b-ffd7d658bbc9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:54.678391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bb960-2055-4194-ad9c-b5e2b98ddd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:54.679320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb960-2055-4194-ad9c-b5e2b98ddd34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.679600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb960-2055-4194-ad9c-b5e2b98ddd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.679927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb960-2055-4194-ad9c-b5e2b98ddd34 admin admin] PATH_INFO: `/v3/projects/3dbd85709a4742f7b13de3ef9fdb067b/users/5b260aa3ffe94b1bb6895b8106e20857/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.696070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bb960-2055-4194-ad9c-b5e2b98ddd34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dbd85709a4742f7b13de3ef9fdb067b, user_id=5b260aa3ffe94b1bb6895b8106e20857, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:54.699327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bb960-2055-4194-ad9c-b5e2b98ddd34 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:54.714854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1046/2085] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:54 2022] PUT /identity/v3/projects/3dbd85709a4742f7b13de3ef9fdb067b/users/5b260aa3ffe94b1bb6895b8106e20857/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:54.726097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49215ab1-b08f-47d9-a867-5d358678e162 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.726458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49215ab1-b08f-47d9-a867-5d358678e162 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.726874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49215ab1-b08f-47d9-a867-5d358678e162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.761718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-49215ab1-b08f-47d9-a867-5d358678e162 None None] MFA Rules not processed for user `5b260aa3ffe94b1bb6895b8106e20857`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:54.762931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-49215ab1-b08f-47d9-a867-5d358678e162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:54.843297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1040/2086] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:23:54.847866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bb960-2055-4194-ad9c-b5e2b98ddd34 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.924410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bb960-2055-4194-ad9c-b5e2b98ddd34 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:54.929070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4d67e-5bc9-43db-bb80-44a1f1ad55c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:54.929856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4d67e-5bc9-43db-bb80-44a1f1ad55c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:54.930136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4d67e-5bc9-43db-bb80-44a1f1ad55c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:54.930412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4d67e-5bc9-43db-bb80-44a1f1ad55c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:54.931125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4d67e-5bc9-43db-bb80-44a1f1ad55c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:54.933008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4d67e-5bc9-43db-bb80-44a1f1ad55c1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:54.960459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1047/2087] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:54 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:54.964509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49215ab1-b08f-47d9-a867-5d358678e162 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:54.975359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49215ab1-b08f-47d9-a867-5d358678e162 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:55.002528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:55.043478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05eb42-1fd1-46ef-b90b-5a75cab86a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:55.044162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05eb42-1fd1-46ef-b90b-5a75cab86a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.044440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05eb42-1fd1-46ef-b90b-5a75cab86a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.044734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05eb42-1fd1-46ef-b90b-5a75cab86a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.045453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05eb42-1fd1-46ef-b90b-5a75cab86a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:55.047370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05eb42-1fd1-46ef-b90b-5a75cab86a49 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:55.072872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1041/2088] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:54 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:55.077159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4d67e-5bc9-43db-bb80-44a1f1ad55c1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:55.088847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4d67e-5bc9-43db-bb80-44a1f1ad55c1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:55.096356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46785bff-6189-4e73-92ef-a59f7d6aa293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:55.097409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46785bff-6189-4e73-92ef-a59f7d6aa293 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.097874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46785bff-6189-4e73-92ef-a59f7d6aa293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.098340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46785bff-6189-4e73-92ef-a59f7d6aa293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.099602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46785bff-6189-4e73-92ef-a59f7d6aa293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:55.102414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46785bff-6189-4e73-92ef-a59f7d6aa293 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:55.117568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1048/2089] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:55.122129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05eb42-1fd1-46ef-b90b-5a75cab86a49 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:55.134948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05eb42-1fd1-46ef-b90b-5a75cab86a49 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:55.140988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b6c6a-97d7-449d-b284-ea0b5d875487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:55.142599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b6c6a-97d7-449d-b284-ea0b5d875487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.142875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b6c6a-97d7-449d-b284-ea0b5d875487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.143282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b6c6a-97d7-449d-b284-ea0b5d875487 admin admin] PATH_INFO: `/v3/projects/d11cc5a0a7dd442e9a5d01b1646cc624/users/0f804a2edcb54166a5ac156a4277c7c6/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.162006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b6c6a-97d7-449d-b284-ea0b5d875487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11cc5a0a7dd442e9a5d01b1646cc624, user_id=0f804a2edcb54166a5ac156a4277c7c6, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:55.164908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b6c6a-97d7-449d-b284-ea0b5d875487 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:55.181908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1042/2090] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:55 2022] PUT /identity/v3/projects/d11cc5a0a7dd442e9a5d01b1646cc624/users/0f804a2edcb54166a5ac156a4277c7c6/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:55.185996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46785bff-6189-4e73-92ef-a59f7d6aa293 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:55.197207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46785bff-6189-4e73-92ef-a59f7d6aa293 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:55.271026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d3144-72b6-4fc4-9cef-b267d1d50a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:55.271906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d3144-72b6-4fc4-9cef-b267d1d50a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.272253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d3144-72b6-4fc4-9cef-b267d1d50a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.272580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d3144-72b6-4fc4-9cef-b267d1d50a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.273486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d3144-72b6-4fc4-9cef-b267d1d50a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:55.276313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d3144-72b6-4fc4-9cef-b267d1d50a3c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:55.290434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1049/2091] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:55.296386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b6c6a-97d7-449d-b284-ea0b5d875487 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:55.310998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b6c6a-97d7-449d-b284-ea0b5d875487 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:55.318752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:55.319895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.320260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.321322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] PATH_INFO: `/v3/projects/d11cc5a0a7dd442e9a5d01b1646cc624/users/0f804a2edcb54166a5ac156a4277c7c6/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.323589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d11cc5a0a7dd442e9a5d01b1646cc624, user_id=0f804a2edcb54166a5ac156a4277c7c6, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:55.326321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:55.338270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1043/2092] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:55 2022] PUT /identity/v3/projects/d11cc5a0a7dd442e9a5d01b1646cc624/users/0f804a2edcb54166a5ac156a4277c7c6/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:55.348816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f98c-9577-451b-8298-c9e31bc16bc5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.349007 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f98c-9577-451b-8298-c9e31bc16bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.349346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f98c-9577-451b-8298-c9e31bc16bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.356907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:55.377710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-04d3f98c-9577-451b-8298-c9e31bc16bc5 None None] MFA Rules not processed for user `0f804a2edcb54166a5ac156a4277c7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:55.378934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-04d3f98c-9577-451b-8298-c9e31bc16bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:55.461230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1050/2093] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:55.473063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:55.475007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:55.543537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e9719-f1f8-402f-ab05-c865f8249a25 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:55.547788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4bd22-079a-4ae5-a2fe-51703171bf33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:55.548473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4bd22-079a-4ae5-a2fe-51703171bf33 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.548751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4bd22-079a-4ae5-a2fe-51703171bf33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.549031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4bd22-079a-4ae5-a2fe-51703171bf33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.559624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4bd22-079a-4ae5-a2fe-51703171bf33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:55.561047 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4bd22-079a-4ae5-a2fe-51703171bf33 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:55.571277 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1044/2094] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:55.703810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3f98c-9577-451b-8298-c9e31bc16bc5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:55.775119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3f98c-9577-451b-8298-c9e31bc16bc5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:55.780343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:55.781393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.781667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.781946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] PATH_INFO: `/v3/users/5b260aa3ffe94b1bb6895b8106e20857` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.783228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b260aa3ffe94b1bb6895b8106e20857)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:55.785154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:55.849878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b260aa3ffe94b1bb6895b8106e20857'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:55.868226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b260aa3ffe94b1bb6895b8106e20857'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:55.871439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b260aa3ffe94b1bb6895b8106e20857'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:55.880144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b260aa3ffe94b1bb6895b8106e20857'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:55.889238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b260aa3ffe94b1bb6895b8106e20857'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:55.896317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1051/2095] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:55 2022] DELETE /identity/v3/users/5b260aa3ffe94b1bb6895b8106e20857 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:55.901391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463ff0-80e7-45fd-909a-8cde6043bb97 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:55.901885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463ff0-80e7-45fd-909a-8cde6043bb97 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:55.901978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3463ff0-80e7-45fd-909a-8cde6043bb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:55.921286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f3463ff0-80e7-45fd-909a-8cde6043bb97 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:55.922609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f3463ff0-80e7-45fd-909a-8cde6043bb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:55.988493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1045/2096] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:55.998321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:55.999730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:56.011006 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:56.068976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab65c2f-9f0c-4312-97ae-f38b7bd65703 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:56.073166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87f9ec-ef07-4808-b33d-48e73bb4a608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:56.073856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87f9ec-ef07-4808-b33d-48e73bb4a608 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:56.074140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87f9ec-ef07-4808-b33d-48e73bb4a608 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:56.074417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87f9ec-ef07-4808-b33d-48e73bb4a608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:56.084728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87f9ec-ef07-4808-b33d-48e73bb4a608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:56.086654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87f9ec-ef07-4808-b33d-48e73bb4a608 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:56.097665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1052/2097] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:23:56.452800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3463ff0-80e7-45fd-909a-8cde6043bb97 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:56.454557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f3463ff0-80e7-45fd-909a-8cde6043bb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:56.473151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3463ff0-80e7-45fd-909a-8cde6043bb97 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:56.483104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:56.484428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:56.484831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:56.485325 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] PATH_INFO: `/v3/projects/3dbd85709a4742f7b13de3ef9fdb067b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:56.487298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dbd85709a4742f7b13de3ef9fdb067b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:56.490550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:56.520403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:56.550015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dbd85709a4742f7b13de3ef9fdb067b', 'request_id': 'req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.557032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dbd85709a4742f7b13de3ef9fdb067b', 'request_id': 'req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.568361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dbd85709a4742f7b13de3ef9fdb067b', 'request_id': 'req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.580864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] The token cache is being invalidate because project 3dbd85709a4742f7b13de3ef9fdb067b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:56.581147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.581824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.583382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1046/2098] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:56 2022] DELETE /identity/v3/projects/3dbd85709a4742f7b13de3ef9fdb067b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:56.587668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87f9ec-ef07-4808-b33d-48e73bb4a608 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:56.598525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87f9ec-ef07-4808-b33d-48e73bb4a608 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:56.667545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:56.668577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:56.669085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:56.669444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] PATH_INFO: `/v3/users/0f804a2edcb54166a5ac156a4277c7c6` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:56.670862 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f804a2edcb54166a5ac156a4277c7c6)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:56.673477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:56.736840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f804a2edcb54166a5ac156a4277c7c6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.750954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f804a2edcb54166a5ac156a4277c7c6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.751763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f804a2edcb54166a5ac156a4277c7c6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.760445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f804a2edcb54166a5ac156a4277c7c6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.771213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0e0d6396-ced7-4536-ae27-bdd12aa4af63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f804a2edcb54166a5ac156a4277c7c6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:56.780612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1053/2099] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:23:56 2022] DELETE /identity/v3/users/0f804a2edcb54166a5ac156a4277c7c6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:57.161837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:57.163420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:57.178247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc2fe22-1d33-4da6-9b88-060d0d810d5c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:57.243459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:57.244442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.244722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.244995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] PATH_INFO: `/v3/projects/d11cc5a0a7dd442e9a5d01b1646cc624` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.246334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d11cc5a0a7dd442e9a5d01b1646cc624)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:57.248313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:57.303171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd11cc5a0a7dd442e9a5d01b1646cc624', 'request_id': 'req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:57.315137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd11cc5a0a7dd442e9a5d01b1646cc624', 'request_id': 'req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:57.322528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd11cc5a0a7dd442e9a5d01b1646cc624', 'request_id': 'req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:57.338990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] The token cache is being invalidate because project d11cc5a0a7dd442e9a5d01b1646cc624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:23:57.339545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:57.340432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-db981c9d-12d9-4298-8ccc-a8ade96ddb5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:23:57.342615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1047/2100] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:23:57 2022] DELETE /identity/v3/projects/d11cc5a0a7dd442e9a5d01b1646cc624 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:57.347789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7689a4-7fe8-449c-b2ae-a07a9fc533b7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.348308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7689a4-7fe8-449c-b2ae-a07a9fc533b7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.348407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7689a4-7fe8-449c-b2ae-a07a9fc533b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.371558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-db7689a4-7fe8-449c-b2ae-a07a9fc533b7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:57.372798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-db7689a4-7fe8-449c-b2ae-a07a9fc533b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:57.405002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:57.456059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1054/2101] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:57.465504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9cfba-ba61-4dd3-91cb-988f9d3101d9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.466695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9cfba-ba61-4dd3-91cb-988f9d3101d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.467091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9cfba-ba61-4dd3-91cb-988f9d3101d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.490804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-73d9cfba-ba61-4dd3-91cb-988f9d3101d9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:57.492042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-73d9cfba-ba61-4dd3-91cb-988f9d3101d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:57.501090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1048/2102] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:57.505422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7689a4-7fe8-449c-b2ae-a07a9fc533b7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:57.515419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7689a4-7fe8-449c-b2ae-a07a9fc533b7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:57.520736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:57.521468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.521753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.522036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.522787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:23:57.523101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:57.525054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:57.537378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1055/2103] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:23:57.541940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9cfba-ba61-4dd3-91cb-988f9d3101d9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:57.552748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9cfba-ba61-4dd3-91cb-988f9d3101d9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:57.558982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1faf029-2fc4-48af-9803-8248e926f680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:57.560084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1faf029-2fc4-48af-9803-8248e926f680 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.560567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1faf029-2fc4-48af-9803-8248e926f680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.561286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1faf029-2fc4-48af-9803-8248e926f680 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.562819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1faf029-2fc4-48af-9803-8248e926f680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:57.564794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1faf029-2fc4-48af-9803-8248e926f680 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:57.605761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1049/2104] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:57 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:57.612043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:57.626605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd2cef-b310-4041-8756-ae7e65d224d0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:57.689318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68893f3-17f7-4bd6-a1a4-5c00215dab39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:57.690057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68893f3-17f7-4bd6-a1a4-5c00215dab39 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.690642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68893f3-17f7-4bd6-a1a4-5c00215dab39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.690924 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68893f3-17f7-4bd6-a1a4-5c00215dab39 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.691636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68893f3-17f7-4bd6-a1a4-5c00215dab39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:57.695121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68893f3-17f7-4bd6-a1a4-5c00215dab39 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:57.733247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1056/2105] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:57 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:57.735644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1faf029-2fc4-48af-9803-8248e926f680 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:57.747435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1faf029-2fc4-48af-9803-8248e926f680 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:57.752923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2158e-37b5-406c-bbf3-e32d9a637a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:57.753623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2158e-37b5-406c-bbf3-e32d9a637a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.753914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2158e-37b5-406c-bbf3-e32d9a637a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.754198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2158e-37b5-406c-bbf3-e32d9a637a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.754942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2158e-37b5-406c-bbf3-e32d9a637a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:57.756586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2158e-37b5-406c-bbf3-e32d9a637a25 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:57.759610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:57.770302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1050/2106] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:57.774500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68893f3-17f7-4bd6-a1a4-5c00215dab39 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:57.785512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68893f3-17f7-4bd6-a1a4-5c00215dab39 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:57.792990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f4170-6091-4737-8e03-a1fa4e3ef518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:57.794152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f4170-6091-4737-8e03-a1fa4e3ef518 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.794495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f4170-6091-4737-8e03-a1fa4e3ef518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.794777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f4170-6091-4737-8e03-a1fa4e3ef518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.795548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f4170-6091-4737-8e03-a1fa4e3ef518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:57.797324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f4170-6091-4737-8e03-a1fa4e3ef518 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:57.811977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1057/2107] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:57.816797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2158e-37b5-406c-bbf3-e32d9a637a25 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:57.828093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2158e-37b5-406c-bbf3-e32d9a637a25 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:57.833308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5408e-d91a-4841-84bf-759285625b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:57.834193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5408e-d91a-4841-84bf-759285625b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.834706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5408e-d91a-4841-84bf-759285625b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.835012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5408e-d91a-4841-84bf-759285625b6e admin admin] PATH_INFO: `/v3/projects/1f2bfbb5261b479b99ca6918432d0f8f/users/196ff115944e43ce925675f44160cba9/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.858241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5408e-d91a-4841-84bf-759285625b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2bfbb5261b479b99ca6918432d0f8f, user_id=196ff115944e43ce925675f44160cba9, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:57.862367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5408e-d91a-4841-84bf-759285625b6e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:57.881721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1051/2108] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:57 2022] PUT /identity/v3/projects/1f2bfbb5261b479b99ca6918432d0f8f/users/196ff115944e43ce925675f44160cba9/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:57.893772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59e892-f127-4e1a-8e21-e9866fb6f049 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:57.894090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59e892-f127-4e1a-8e21-e9866fb6f049 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:57.894371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59e892-f127-4e1a-8e21-e9866fb6f049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:57.926745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7e59e892-f127-4e1a-8e21-e9866fb6f049 None None] MFA Rules not processed for user `196ff115944e43ce925675f44160cba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:57.927984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7e59e892-f127-4e1a-8e21-e9866fb6f049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:58.012658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1058/2109] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:58.014012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5408e-d91a-4841-84bf-759285625b6e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:58.088476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5408e-d91a-4841-84bf-759285625b6e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:58.094228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404817f0-8710-4746-b478-9aed1d4c57bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:58.094902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817f0-8710-4746-b478-9aed1d4c57bc admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.095188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817f0-8710-4746-b478-9aed1d4c57bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.095478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404817f0-8710-4746-b478-9aed1d4c57bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.096195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404817f0-8710-4746-b478-9aed1d4c57bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:58.098188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404817f0-8710-4746-b478-9aed1d4c57bc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:58.131865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1052/2110] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:23:58 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:58.136248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59e892-f127-4e1a-8e21-e9866fb6f049 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:58.147433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59e892-f127-4e1a-8e21-e9866fb6f049 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:58.174469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:58.213668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b8293-4606-4370-979f-08c547547c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:58.214367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b8293-4606-4370-979f-08c547547c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.214649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b8293-4606-4370-979f-08c547547c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.214922 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b8293-4606-4370-979f-08c547547c88 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.215626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b8293-4606-4370-979f-08c547547c88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:58.217613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b8293-4606-4370-979f-08c547547c88 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:58.242473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1059/2111] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:23:58 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:23:58.247045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404817f0-8710-4746-b478-9aed1d4c57bc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:58.258485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404817f0-8710-4746-b478-9aed1d4c57bc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:58.264647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb82a2-49d3-4ee1-9eb6-b528cef8497b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:58.265305 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb82a2-49d3-4ee1-9eb6-b528cef8497b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.265588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb82a2-49d3-4ee1-9eb6-b528cef8497b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.265862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb82a2-49d3-4ee1-9eb6-b528cef8497b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.266641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb82a2-49d3-4ee1-9eb6-b528cef8497b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:58.268424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb82a2-49d3-4ee1-9eb6-b528cef8497b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:58.281254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1053/2112] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:58.285924 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b8293-4606-4370-979f-08c547547c88 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:58.297223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b8293-4606-4370-979f-08c547547c88 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:58.303423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189e573-77ff-4bd4-b01d-efbfa022d305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:58.304343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189e573-77ff-4bd4-b01d-efbfa022d305 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.304802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189e573-77ff-4bd4-b01d-efbfa022d305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.305441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189e573-77ff-4bd4-b01d-efbfa022d305 admin admin] PATH_INFO: `/v3/projects/ea8cd34bc01848e6bd12dcc4a2976d50/users/2b49c1542f0244ae829262c16b155895/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.325550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189e573-77ff-4bd4-b01d-efbfa022d305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8cd34bc01848e6bd12dcc4a2976d50, user_id=2b49c1542f0244ae829262c16b155895, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:58.328637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189e573-77ff-4bd4-b01d-efbfa022d305 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:58.347494 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1060/2113] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:58 2022] PUT /identity/v3/projects/ea8cd34bc01848e6bd12dcc4a2976d50/users/2b49c1542f0244ae829262c16b155895/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:23:58.348756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb82a2-49d3-4ee1-9eb6-b528cef8497b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:58.361231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb82a2-49d3-4ee1-9eb6-b528cef8497b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:58.427787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128a2e1-4df2-4029-b12e-12e330155ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:58.428465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128a2e1-4df2-4029-b12e-12e330155ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.428834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128a2e1-4df2-4029-b12e-12e330155ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.429017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128a2e1-4df2-4029-b12e-12e330155ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.430017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128a2e1-4df2-4029-b12e-12e330155ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:58.431641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128a2e1-4df2-4029-b12e-12e330155ec6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:58.441792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1054/2114] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:23:58.446144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189e573-77ff-4bd4-b01d-efbfa022d305 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:58.457435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189e573-77ff-4bd4-b01d-efbfa022d305 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:58.462269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:58.463192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.463476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.463763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] PATH_INFO: `/v3/projects/ea8cd34bc01848e6bd12dcc4a2976d50/users/2b49c1542f0244ae829262c16b155895/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.465830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8cd34bc01848e6bd12dcc4a2976d50, user_id=2b49c1542f0244ae829262c16b155895, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:58.468689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:58.480437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1061/2115] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:23:58 2022] PUT /identity/v3/projects/ea8cd34bc01848e6bd12dcc4a2976d50/users/2b49c1542f0244ae829262c16b155895/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:23:58.492015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d00ae8-f37a-48d0-b85f-4ed44e015cce None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.492369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d00ae8-f37a-48d0-b85f-4ed44e015cce None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.492649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d00ae8-f37a-48d0-b85f-4ed44e015cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.523067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-22d00ae8-f37a-48d0-b85f-4ed44e015cce None None] MFA Rules not processed for user `2b49c1542f0244ae829262c16b155895`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:23:58.524451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-22d00ae8-f37a-48d0-b85f-4ed44e015cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:58.564735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:23:58.616887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1055/2116] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:23:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:23:58.631268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:58.633541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:58.706183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811b34cc-173b-4c73-8090-08afc797e82e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:58.710583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:58.711432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.711845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.712161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.723217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:58.724693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:58.738593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1062/2117] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:23:58.840092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d00ae8-f37a-48d0-b85f-4ed44e015cce None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:23:58.841773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-22d00ae8-f37a-48d0-b85f-4ed44e015cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:23:58.859281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d00ae8-f37a-48d0-b85f-4ed44e015cce None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:23:58.864187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0234a04b-80cb-4dee-a97d-d0b0327684d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:23:58.865146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234a04b-80cb-4dee-a97d-d0b0327684d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:23:58.865632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234a04b-80cb-4dee-a97d-d0b0327684d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:23:58.866076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0234a04b-80cb-4dee-a97d-d0b0327684d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:23:58.931672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234a04b-80cb-4dee-a97d-d0b0327684d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:23:58.934460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0234a04b-80cb-4dee-a97d-d0b0327684d5 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:23:58.946350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1056/2118] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:23:58 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:03.454261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:03.455746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:03.532686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234f799-fc96-4fb0-8189-c29db4bece97 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:03.537279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:03.538153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:03.538433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:03.538716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] PATH_INFO: `/v3/users/2a7657c174ec40ba806f3851b6f1435c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:03.540058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a7657c174ec40ba806f3851b6f1435c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:03.541976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:03.566818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:03.595664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a7657c174ec40ba806f3851b6f1435c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:03.616128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a7657c174ec40ba806f3851b6f1435c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:03.616925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a7657c174ec40ba806f3851b6f1435c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:03.627818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a7657c174ec40ba806f3851b6f1435c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:03.639446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a7657c174ec40ba806f3851b6f1435c'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:03.653250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1063/2119] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:03 2022] DELETE /identity/v3/users/2a7657c174ec40ba806f3851b6f1435c => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:03.656021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472fe6-6f7b-44c9-a6b6-32ff3488e9ab None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:03.656630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472fe6-6f7b-44c9-a6b6-32ff3488e9ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:03.656630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0472fe6-6f7b-44c9-a6b6-32ff3488e9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:03.676293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b0472fe6-6f7b-44c9-a6b6-32ff3488e9ab None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:03.677998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b0472fe6-6f7b-44c9-a6b6-32ff3488e9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:03.756554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1057/2120] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:03.767354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:03.768945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:03.840684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc3e2c-9a89-4005-bf75-ecdbcede0ded admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:03.845246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae10611-0838-4419-bf40-9f6d5492d3ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:03.846280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae10611-0838-4419-bf40-9f6d5492d3ea service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:03.846830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae10611-0838-4419-bf40-9f6d5492d3ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:03.846830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae10611-0838-4419-bf40-9f6d5492d3ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:03.860188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae10611-0838-4419-bf40-9f6d5492d3ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:03.861679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae10611-0838-4419-bf40-9f6d5492d3ea service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:03.872356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1064/2121] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:24:04.180897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0472fe6-6f7b-44c9-a6b6-32ff3488e9ab None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:04.182742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b0472fe6-6f7b-44c9-a6b6-32ff3488e9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:04.201418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0472fe6-6f7b-44c9-a6b6-32ff3488e9ab None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:04.207217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:04.208035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.208328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.208605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] PATH_INFO: `/v3/projects/bd98cf76b6574417895e5f535e26bb07` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.210132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd98cf76b6574417895e5f535e26bb07)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:04.212932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:04.270993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd98cf76b6574417895e5f535e26bb07', 'request_id': 'req-75d8450a-adba-417d-9a25-14b9f8b3780e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:04.276958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd98cf76b6574417895e5f535e26bb07', 'request_id': 'req-75d8450a-adba-417d-9a25-14b9f8b3780e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:04.282419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:04.290121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd98cf76b6574417895e5f535e26bb07', 'request_id': 'req-75d8450a-adba-417d-9a25-14b9f8b3780e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:04.309747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] The token cache is being invalidate because project bd98cf76b6574417895e5f535e26bb07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:04.310081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:04.311250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-75d8450a-adba-417d-9a25-14b9f8b3780e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:04.313375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1058/2122] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:04 2022] DELETE /identity/v3/projects/bd98cf76b6574417895e5f535e26bb07 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:04.318123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e550-32a1-42d8-8c17-69483ded1056 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.318426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e550-32a1-42d8-8c17-69483ded1056 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.318699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d3e550-32a1-42d8-8c17-69483ded1056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.343589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-40d3e550-32a1-42d8-8c17-69483ded1056 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:04.344887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-40d3e550-32a1-42d8-8c17-69483ded1056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:04.420173 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1065/2123] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:04.428740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339810-fbe4-48cf-9c33-09641b2b8959 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.429049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339810-fbe4-48cf-9c33-09641b2b8959 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.429380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa339810-fbe4-48cf-9c33-09641b2b8959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.453342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fa339810-fbe4-48cf-9c33-09641b2b8959 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:04.454682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fa339810-fbe4-48cf-9c33-09641b2b8959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:04.463816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1059/2124] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:04.468261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e550-32a1-42d8-8c17-69483ded1056 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:04.479058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d3e550-32a1-42d8-8c17-69483ded1056 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:04.484323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:04.485044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.485409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.485692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.490063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:04.490063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:04.490063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:04.505163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1066/2125] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:04.506669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa339810-fbe4-48cf-9c33-09641b2b8959 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:04.516874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa339810-fbe4-48cf-9c33-09641b2b8959 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:04.521811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7d0f-3d7c-412c-bb39-3cdac6c2d1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:04.522486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7d0f-3d7c-412c-bb39-3cdac6c2d1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.522783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7d0f-3d7c-412c-bb39-3cdac6c2d1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.523065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e7d0f-3d7c-412c-bb39-3cdac6c2d1f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.523919 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e7d0f-3d7c-412c-bb39-3cdac6c2d1f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:04.526115 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e7d0f-3d7c-412c-bb39-3cdac6c2d1f6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:04.556941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1060/2126] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:04 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:04.561538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:04.572193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5378dcb7-33f9-4149-983b-58e026e2e8e2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:04.606485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:04.643034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5670-4c66-462c-8407-2dbd75f0736e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:04.643737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5670-4c66-462c-8407-2dbd75f0736e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.644026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5670-4c66-462c-8407-2dbd75f0736e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.644308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf5670-4c66-462c-8407-2dbd75f0736e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.645057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5670-4c66-462c-8407-2dbd75f0736e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:04.647034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf5670-4c66-462c-8407-2dbd75f0736e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:04.669833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1067/2127] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:04 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:04.674469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7d0f-3d7c-412c-bb39-3cdac6c2d1f6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:04.685517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e7d0f-3d7c-412c-bb39-3cdac6c2d1f6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:04.690748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1bde0-f944-44de-bdd0-b99aeff72400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:04.691426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1bde0-f944-44de-bdd0-b99aeff72400 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.691865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1bde0-f944-44de-bdd0-b99aeff72400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.692141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1bde0-f944-44de-bdd0-b99aeff72400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.692869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1bde0-f944-44de-bdd0-b99aeff72400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:04.694538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb1bde0-f944-44de-bdd0-b99aeff72400 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:04.706497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1061/2128] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:04.710551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5670-4c66-462c-8407-2dbd75f0736e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:04.721141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf5670-4c66-462c-8407-2dbd75f0736e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:04.725961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaedf9-9922-4add-8d65-b851369455fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:04.726907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaedf9-9922-4add-8d65-b851369455fe admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.727074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaedf9-9922-4add-8d65-b851369455fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.727256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaedf9-9922-4add-8d65-b851369455fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.728167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaedf9-9922-4add-8d65-b851369455fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:04.729827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaedf9-9922-4add-8d65-b851369455fe admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:04.742120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1068/2129] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:24:04.746722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1bde0-f944-44de-bdd0-b99aeff72400 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:04.757498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb1bde0-f944-44de-bdd0-b99aeff72400 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:04.764303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5928475-8a36-4474-96e8-be0dac164500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:04.765364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5928475-8a36-4474-96e8-be0dac164500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.765750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5928475-8a36-4474-96e8-be0dac164500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.766070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5928475-8a36-4474-96e8-be0dac164500 admin admin] PATH_INFO: `/v3/projects/79128a84c79e4690bec90d0bfc838e09/users/760eb98fa66947378c33f79a856e22f7/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.784811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5928475-8a36-4474-96e8-be0dac164500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79128a84c79e4690bec90d0bfc838e09, user_id=760eb98fa66947378c33f79a856e22f7, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:04.788029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5928475-8a36-4474-96e8-be0dac164500 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:04.804912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1062/2130] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:04 2022] PUT /identity/v3/projects/79128a84c79e4690bec90d0bfc838e09/users/760eb98fa66947378c33f79a856e22f7/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:04.826989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4682be-21a5-40f6-93f0-da1d80fb1760 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:04.827309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4682be-21a5-40f6-93f0-da1d80fb1760 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:04.829079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4682be-21a5-40f6-93f0-da1d80fb1760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:04.859445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1e4682be-21a5-40f6-93f0-da1d80fb1760 None None] MFA Rules not processed for user `760eb98fa66947378c33f79a856e22f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:04.860793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1e4682be-21a5-40f6-93f0-da1d80fb1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:04.936917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1069/2131] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:04.944407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5928475-8a36-4474-96e8-be0dac164500 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:05.025872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5928475-8a36-4474-96e8-be0dac164500 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:05.031089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce8a3-9bf7-40ce-8663-8776f5ce4340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:05.031854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce8a3-9bf7-40ce-8663-8776f5ce4340 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:05.032198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce8a3-9bf7-40ce-8663-8776f5ce4340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:05.032564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ce8a3-9bf7-40ce-8663-8776f5ce4340 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:05.045872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce8a3-9bf7-40ce-8663-8776f5ce4340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:05.045872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ce8a3-9bf7-40ce-8663-8776f5ce4340 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:05.074705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1063/2132] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:04 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:05.079404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4682be-21a5-40f6-93f0-da1d80fb1760 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:05.092858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4682be-21a5-40f6-93f0-da1d80fb1760 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:05.216309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0258a6-bba1-46ac-ab7d-45a24ac694ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:05.217554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0258a6-bba1-46ac-ab7d-45a24ac694ea admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:05.218045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0258a6-bba1-46ac-ab7d-45a24ac694ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:05.218524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0258a6-bba1-46ac-ab7d-45a24ac694ea admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:05.220402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0258a6-bba1-46ac-ab7d-45a24ac694ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:05.223683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0258a6-bba1-46ac-ab7d-45a24ac694ea admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:05.281691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1070/2133] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:05 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:05.290782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce8a3-9bf7-40ce-8663-8776f5ce4340 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:05.306483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ce8a3-9bf7-40ce-8663-8776f5ce4340 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:05.316745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3abd61-d7ec-4878-ab69-3238cc5f6d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:05.318347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3abd61-d7ec-4878-ab69-3238cc5f6d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:05.319612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3abd61-d7ec-4878-ab69-3238cc5f6d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:05.319612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3abd61-d7ec-4878-ab69-3238cc5f6d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:05.319612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3abd61-d7ec-4878-ab69-3238cc5f6d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:05.322288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3abd61-d7ec-4878-ab69-3238cc5f6d6a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:05.337276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1064/2134] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:05.340972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0258a6-bba1-46ac-ab7d-45a24ac694ea admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:05.353017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0258a6-bba1-46ac-ab7d-45a24ac694ea admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:05.359272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc99fda-cb55-4c81-b0dd-96b4e4b29d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:05.360078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc99fda-cb55-4c81-b0dd-96b4e4b29d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:05.360427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc99fda-cb55-4c81-b0dd-96b4e4b29d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:05.361054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc99fda-cb55-4c81-b0dd-96b4e4b29d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:05.361806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc99fda-cb55-4c81-b0dd-96b4e4b29d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:05.363545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc99fda-cb55-4c81-b0dd-96b4e4b29d54 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:05.383299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1071/2135] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:24:05.388926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3abd61-d7ec-4878-ab69-3238cc5f6d6a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:05.402082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3abd61-d7ec-4878-ab69-3238cc5f6d6a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:05.408405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2ec13-5894-4fd3-b863-fb284d7ac541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:05.409361 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ec13-5894-4fd3-b863-fb284d7ac541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:05.409768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ec13-5894-4fd3-b863-fb284d7ac541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:05.410066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ec13-5894-4fd3-b863-fb284d7ac541 admin admin] PATH_INFO: `/v3/projects/327b87f2c7fb4fa5b5e6e34b48a1bf38/users/2f9a34806fb042518691211815916ccb/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:05.439780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2ec13-5894-4fd3-b863-fb284d7ac541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=327b87f2c7fb4fa5b5e6e34b48a1bf38, user_id=2f9a34806fb042518691211815916ccb, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:05.446562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2ec13-5894-4fd3-b863-fb284d7ac541 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:05.481078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1065/2136] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:05 2022] PUT /identity/v3/projects/327b87f2c7fb4fa5b5e6e34b48a1bf38/users/2f9a34806fb042518691211815916ccb/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:05.484336 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497c5f-6321-4803-aefb-edaaab005ff6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:05.493381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497c5f-6321-4803-aefb-edaaab005ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:05.494058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99497c5f-6321-4803-aefb-edaaab005ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:05.531841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-99497c5f-6321-4803-aefb-edaaab005ff6 None None] MFA Rules not processed for user `2f9a34806fb042518691211815916ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:05.534207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-99497c5f-6321-4803-aefb-edaaab005ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:05.542500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:05.629570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1072/2137] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:05.635033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2ec13-5894-4fd3-b863-fb284d7ac541 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:05.707268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:05.740781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2ec13-5894-4fd3-b863-fb284d7ac541 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:05.747180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293f134-85a7-4aa9-bc6f-b9d19d94192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:05.747875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293f134-85a7-4aa9-bc6f-b9d19d94192f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:05.748171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293f134-85a7-4aa9-bc6f-b9d19d94192f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:05.748440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293f134-85a7-4aa9-bc6f-b9d19d94192f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:05.749231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293f134-85a7-4aa9-bc6f-b9d19d94192f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:05.751306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293f134-85a7-4aa9-bc6f-b9d19d94192f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:05.807438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1066/2138] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:05 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:05.816692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99497c5f-6321-4803-aefb-edaaab005ff6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:05.836067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99497c5f-6321-4803-aefb-edaaab005ff6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:05.923915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219dc446-35ce-4295-aeac-be2b39d918e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:05.924620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219dc446-35ce-4295-aeac-be2b39d918e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:05.924902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219dc446-35ce-4295-aeac-be2b39d918e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:05.925218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219dc446-35ce-4295-aeac-be2b39d918e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:05.926136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219dc446-35ce-4295-aeac-be2b39d918e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:05.929338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219dc446-35ce-4295-aeac-be2b39d918e6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:05.966697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1073/2139] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:05 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:05.972793 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293f134-85a7-4aa9-bc6f-b9d19d94192f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:05.992443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293f134-85a7-4aa9-bc6f-b9d19d94192f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:06.003717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a25c1-900f-48ef-871f-bf5f18b67d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:06.004717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a25c1-900f-48ef-871f-bf5f18b67d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:06.005248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a25c1-900f-48ef-871f-bf5f18b67d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:06.005652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a25c1-900f-48ef-871f-bf5f18b67d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:06.006790 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a25c1-900f-48ef-871f-bf5f18b67d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:06.009479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a25c1-900f-48ef-871f-bf5f18b67d25 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:06.033322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1067/2140] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:06.034869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219dc446-35ce-4295-aeac-be2b39d918e6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:06.051732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219dc446-35ce-4295-aeac-be2b39d918e6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:06.061873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbce5f4-f4c4-4203-8d8e-383baaaaad06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:06.063153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbce5f4-f4c4-4203-8d8e-383baaaaad06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:06.063559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbce5f4-f4c4-4203-8d8e-383baaaaad06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:06.064059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbce5f4-f4c4-4203-8d8e-383baaaaad06 admin admin] PATH_INFO: `/v3/projects/5009ae5345d442db98c791c7989bf38a/users/c6a2b185755d4472830abeaab149e44f/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:06.092256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbce5f4-f4c4-4203-8d8e-383baaaaad06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5009ae5345d442db98c791c7989bf38a, user_id=c6a2b185755d4472830abeaab149e44f, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:06.095659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbce5f4-f4c4-4203-8d8e-383baaaaad06 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:06.127150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1074/2141] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:06 2022] PUT /identity/v3/projects/5009ae5345d442db98c791c7989bf38a/users/c6a2b185755d4472830abeaab149e44f/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:06.133558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a25c1-900f-48ef-871f-bf5f18b67d25 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:06.162458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a25c1-900f-48ef-871f-bf5f18b67d25 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:06.281434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884ff64-7e81-419d-a2aa-43d17f095ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:06.282614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884ff64-7e81-419d-a2aa-43d17f095ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:06.283120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884ff64-7e81-419d-a2aa-43d17f095ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:06.283679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884ff64-7e81-419d-a2aa-43d17f095ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:06.285044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884ff64-7e81-419d-a2aa-43d17f095ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:06.288491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884ff64-7e81-419d-a2aa-43d17f095ba6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:06.313970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1068/2142] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:06.314978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbce5f4-f4c4-4203-8d8e-383baaaaad06 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:06.332078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbce5f4-f4c4-4203-8d8e-383baaaaad06 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:06.338293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:06.339767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:06.340235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:06.340693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] PATH_INFO: `/v3/projects/5009ae5345d442db98c791c7989bf38a/users/c6a2b185755d4472830abeaab149e44f/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:06.343171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5009ae5345d442db98c791c7989bf38a, user_id=c6a2b185755d4472830abeaab149e44f, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:06.346861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:06.368748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1075/2143] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:06 2022] PUT /identity/v3/projects/5009ae5345d442db98c791c7989bf38a/users/c6a2b185755d4472830abeaab149e44f/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:06.385966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bf750-5645-40dd-b512-63343afea34d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:06.386663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bf750-5645-40dd-b512-63343afea34d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:06.387776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bf750-5645-40dd-b512-63343afea34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:06.441638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f96bf750-5645-40dd-b512-63343afea34d None None] MFA Rules not processed for user `c6a2b185755d4472830abeaab149e44f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:06.444160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f96bf750-5645-40dd-b512-63343afea34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:06.600004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1069/2144] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:06.618293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:06.622354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:06.767409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0a475-35eb-4286-b998-9cb4446febbf admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:06.773710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909455ef-d06a-448b-bde3-693c5b0bf40e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:06.775086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909455ef-d06a-448b-bde3-693c5b0bf40e service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:06.775602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909455ef-d06a-448b-bde3-693c5b0bf40e service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:06.776089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909455ef-d06a-448b-bde3-693c5b0bf40e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:06.802926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:06.921117 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909455ef-d06a-448b-bde3-693c5b0bf40e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:06.923312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909455ef-d06a-448b-bde3-693c5b0bf40e service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:06.943991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1076/2145] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:08.499154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0bf7-86ae-48b6-8649-0b6d922bfaa1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:08.499710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0bf7-86ae-48b6-8649-0b6d922bfaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:08.499812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0bf7-86ae-48b6-8649-0b6d922bfaa1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:08.501402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1070/2146] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:08.535959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4dcb5-a81a-4227-b94f-8b77f8019559 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:08.536537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4dcb5-a81a-4227-b94f-8b77f8019559 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:08.536537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4dcb5-a81a-4227-b94f-8b77f8019559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:08.557473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d4d4dcb5-a81a-4227-b94f-8b77f8019559 None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:08.558760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d4d4dcb5-a81a-4227-b94f-8b77f8019559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:08.566837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb0bf7-86ae-48b6-8649-0b6d922bfaa1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:08.568713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-18bb0bf7-86ae-48b6-8649-0b6d922bfaa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:08.611434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:08.645450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1077/2147] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:24:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:08.656023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb0bf7-86ae-48b6-8649-0b6d922bfaa1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:08.658328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4dcb5-a81a-4227-b94f-8b77f8019559 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:08.659825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d4d4dcb5-a81a-4227-b94f-8b77f8019559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:08.661809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:08.663469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:08.664148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:08.664752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] PATH_INFO: `/v3/users/a132fd2fe24343239eafa8c46abaa2ed` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:08.666984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a132fd2fe24343239eafa8c46abaa2ed)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:08.669301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:08.675955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4dcb5-a81a-4227-b94f-8b77f8019559 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:08.682268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85c41a-d39f-439e-b212-49163d03500b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:08.682940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85c41a-d39f-439e-b212-49163d03500b service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:08.683226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85c41a-d39f-439e-b212-49163d03500b service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:08.683523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85c41a-d39f-439e-b212-49163d03500b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:08.704250 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85c41a-d39f-439e-b212-49163d03500b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:08.706078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85c41a-d39f-439e-b212-49163d03500b service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:08.721082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1078/2148] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:24:08.754753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a132fd2fe24343239eafa8c46abaa2ed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:08.755419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a132fd2fe24343239eafa8c46abaa2ed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:08.756252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a132fd2fe24343239eafa8c46abaa2ed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:08.778429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a132fd2fe24343239eafa8c46abaa2ed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:08.789222 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6a5030d5-cec1-4b93-b137-7d3cddc9720a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a132fd2fe24343239eafa8c46abaa2ed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:08.809675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1071/2149] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:08 2022] DELETE /identity/v3/users/a132fd2fe24343239eafa8c46abaa2ed => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:24:09.025721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85c41a-d39f-439e-b212-49163d03500b service placement] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:09.037814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85c41a-d39f-439e-b212-49163d03500b service placement] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:09.102775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:09.103463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.103743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.104042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] PATH_INFO: `/v3/projects/859da351f9cc437ebe79e5bdb87a177d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.105369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=859da351f9cc437ebe79e5bdb87a177d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:09.107628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:09.166554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:09.175021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '859da351f9cc437ebe79e5bdb87a177d', 'request_id': 'req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:09.185170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '859da351f9cc437ebe79e5bdb87a177d', 'request_id': 'req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:09.192538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '859da351f9cc437ebe79e5bdb87a177d', 'request_id': 'req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:09.203636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] The token cache is being invalidate because project 859da351f9cc437ebe79e5bdb87a177d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:09.203923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:09.204584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f4e18be7-fcc5-4750-bdb8-6afd0a839b4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:09.206390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1079/2150] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:09 2022] DELETE /identity/v3/projects/859da351f9cc437ebe79e5bdb87a177d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:09.212534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b7de-4fa6-451c-8ea5-5cacadf776c7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.212841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b7de-4fa6-451c-8ea5-5cacadf776c7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.213116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb87b7de-4fa6-451c-8ea5-5cacadf776c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.232972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bb87b7de-4fa6-451c-8ea5-5cacadf776c7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:09.234242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb87b7de-4fa6-451c-8ea5-5cacadf776c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:09.302224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1072/2151] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:09.311500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe97d4-c807-4b3d-bbe8-705c513eecf7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.312022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe97d4-c807-4b3d-bbe8-705c513eecf7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.312355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe97d4-c807-4b3d-bbe8-705c513eecf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.332837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7cfe97d4-c807-4b3d-bbe8-705c513eecf7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:09.334075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7cfe97d4-c807-4b3d-bbe8-705c513eecf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:09.343774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1080/2152] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:09.348506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87b7de-4fa6-451c-8ea5-5cacadf776c7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:09.361921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb87b7de-4fa6-451c-8ea5-5cacadf776c7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:09.368568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:09.369828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.370108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.370380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.371122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:09.371424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:09.373072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:09.390069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1073/2153] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:09.394742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe97d4-c807-4b3d-bbe8-705c513eecf7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:09.406566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe97d4-c807-4b3d-bbe8-705c513eecf7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:09.411919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870afd1-3320-4da6-b00f-2dc5a4c14a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:09.412527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870afd1-3320-4da6-b00f-2dc5a4c14a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.412955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870afd1-3320-4da6-b00f-2dc5a4c14a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.413292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870afd1-3320-4da6-b00f-2dc5a4c14a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.414443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870afd1-3320-4da6-b00f-2dc5a4c14a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:09.417129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870afd1-3320-4da6-b00f-2dc5a4c14a9e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:09.451696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1081/2154] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:09 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:09.456010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:09.469649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041d0a6-fa42-48c2-a6ac-f65874f5acad admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:09.517103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:09.540797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55801941-9600-4581-aae5-26b5f59f08d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:09.541543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55801941-9600-4581-aae5-26b5f59f08d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.541833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55801941-9600-4581-aae5-26b5f59f08d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.542155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55801941-9600-4581-aae5-26b5f59f08d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.542829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55801941-9600-4581-aae5-26b5f59f08d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:09.544971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55801941-9600-4581-aae5-26b5f59f08d3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:09.578714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1074/2155] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:09 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:09.582616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870afd1-3320-4da6-b00f-2dc5a4c14a9e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:09.596082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870afd1-3320-4da6-b00f-2dc5a4c14a9e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:09.600980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686fc68-c69c-4ea0-b539-4d40950cd7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:09.601713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686fc68-c69c-4ea0-b539-4d40950cd7b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.601993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686fc68-c69c-4ea0-b539-4d40950cd7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.602270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686fc68-c69c-4ea0-b539-4d40950cd7b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.603010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686fc68-c69c-4ea0-b539-4d40950cd7b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:09.604599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8686fc68-c69c-4ea0-b539-4d40950cd7b3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:09.616944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1082/2156] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:09.623694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55801941-9600-4581-aae5-26b5f59f08d3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:09.639478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55801941-9600-4581-aae5-26b5f59f08d3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:09.646875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cb72c7-723f-4c0a-9f00-55d4084eac66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:09.647961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb72c7-723f-4c0a-9f00-55d4084eac66 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.648429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb72c7-723f-4c0a-9f00-55d4084eac66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.648826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cb72c7-723f-4c0a-9f00-55d4084eac66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.649682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb72c7-723f-4c0a-9f00-55d4084eac66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:09.651721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cb72c7-723f-4c0a-9f00-55d4084eac66 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:09.664040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1075/2157] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:09.670734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686fc68-c69c-4ea0-b539-4d40950cd7b3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:09.680534 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8686fc68-c69c-4ea0-b539-4d40950cd7b3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:09.686703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:09.687617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.687880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.688156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] PATH_INFO: `/v3/projects/489df5c12d274a23be1c0bf87dc09dc7/users/b8c37bfdb2384b838f6e5a0c7b0c1b4d/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.705867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=489df5c12d274a23be1c0bf87dc09dc7, user_id=b8c37bfdb2384b838f6e5a0c7b0c1b4d, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:09.708655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:09.726120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1083/2158] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:09 2022] PUT /identity/v3/projects/489df5c12d274a23be1c0bf87dc09dc7/users/b8c37bfdb2384b838f6e5a0c7b0c1b4d/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:09.738779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43d34-2571-4123-8198-54797e39991f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.739083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43d34-2571-4123-8198-54797e39991f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.739364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43d34-2571-4123-8198-54797e39991f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.772436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f2d43d34-2571-4123-8198-54797e39991f None None] MFA Rules not processed for user `b8c37bfdb2384b838f6e5a0c7b0c1b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:09.773930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f2d43d34-2571-4123-8198-54797e39991f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:09.867315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1076/2159] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:09.882075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:09.884624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:09.979578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5e2de-f17a-4cc6-977d-30fb29bed912 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:09.987148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:09.987804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:09.988087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:09.988358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:09.999936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:10.002223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:10.015401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1084/2160] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:24:10.568322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d43d34-2571-4123-8198-54797e39991f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:10.569810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f2d43d34-2571-4123-8198-54797e39991f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:10.653853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d43d34-2571-4123-8198-54797e39991f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:10.661860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:10.661860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:10.661860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:10.662362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] PATH_INFO: `/v3/users/196ff115944e43ce925675f44160cba9` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:10.665377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=196ff115944e43ce925675f44160cba9)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:10.668359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:10.749554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196ff115944e43ce925675f44160cba9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:10.757644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196ff115944e43ce925675f44160cba9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:10.758782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196ff115944e43ce925675f44160cba9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:10.761507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:10.779466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196ff115944e43ce925675f44160cba9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:10.793883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc60aa37-8412-4bac-8ccb-1e8523656be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196ff115944e43ce925675f44160cba9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:10.814100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1077/2161] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:10 2022] DELETE /identity/v3/users/196ff115944e43ce925675f44160cba9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:10.817748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:10.819486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43398f10-f820-4808-bf16-156af16055f4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:10.819844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43398f10-f820-4808-bf16-156af16055f4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:10.820082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:10.820202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43398f10-f820-4808-bf16-156af16055f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:10.842489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-43398f10-f820-4808-bf16-156af16055f4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:10.843758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43398f10-f820-4808-bf16-156af16055f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:10.885109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:10.890872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8314fbd-ca3a-41cb-8080-b75f2c5a0f89 service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:10.935659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1078/2162] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:10.946995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43398f10-f820-4808-bf16-156af16055f4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:10.949008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43398f10-f820-4808-bf16-156af16055f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:10.962622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:10.963280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:10.963564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:10.963843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:10.966212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:11.015298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43398f10-f820-4808-bf16-156af16055f4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:11.020056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:11.020969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:11.021465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:11.021959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:11.032868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:11.034358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:11.045062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:11.046547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:11.048867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1079/2163] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:11.059418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1085/2164] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:11.364414 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:11.365867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:11.380161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1315b8-d5d3-4aa5-99bf-3fbf00fd6d7f service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:11.385099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:11.386008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:11.386287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:11.386561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] PATH_INFO: `/v3/projects/1f2bfbb5261b479b99ca6918432d0f8f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:11.387773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2bfbb5261b479b99ca6918432d0f8f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:11.389860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:11.442163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2bfbb5261b479b99ca6918432d0f8f', 'request_id': 'req-b2255113-bea2-4aa8-9aab-ec714d05f06d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.449553 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2bfbb5261b479b99ca6918432d0f8f', 'request_id': 'req-b2255113-bea2-4aa8-9aab-ec714d05f06d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.457336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2bfbb5261b479b99ca6918432d0f8f', 'request_id': 'req-b2255113-bea2-4aa8-9aab-ec714d05f06d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.458732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:11.475650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] The token cache is being invalidate because project 1f2bfbb5261b479b99ca6918432d0f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:11.475848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.478917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.482928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1080/2165] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:11 2022] DELETE /identity/v3/projects/1f2bfbb5261b479b99ca6918432d0f8f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:11.489116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:11.503529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a3a30-8cbf-43c5-be06-4956a255f1f7 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:11.593602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:11.594769 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:11.595310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:11.595751 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] PATH_INFO: `/v3/users/2b49c1542f0244ae829262c16b155895` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:11.597452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b49c1542f0244ae829262c16b155895)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:11.599412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:11.660954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b49c1542f0244ae829262c16b155895'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.674799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:11.677813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:11.678307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b49c1542f0244ae829262c16b155895'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.679120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b49c1542f0244ae829262c16b155895'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.680051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:11.691318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b49c1542f0244ae829262c16b155895'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.702112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b49c1542f0244ae829262c16b155895'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:11.712601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1086/2166] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:11 2022] DELETE /identity/v3/users/2b49c1542f0244ae829262c16b155895 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:11.752597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2255113-bea2-4aa8-9aab-ec714d05f06d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:11.828809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20d23a-0f1c-4d26-957c-f0bc2c751587 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:11.829509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d23a-0f1c-4d26-957c-f0bc2c751587 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:11.829792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d23a-0f1c-4d26-957c-f0bc2c751587 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:11.830069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20d23a-0f1c-4d26-957c-f0bc2c751587 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:11.832870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dc20d23a-0f1c-4d26-957c-f0bc2c751587 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:11.934786 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20d23a-0f1c-4d26-957c-f0bc2c751587 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:11.936604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20d23a-0f1c-4d26-957c-f0bc2c751587 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:11.949505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1081/2167] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:12.156180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:12.157595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:12.234944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfc833-994d-41c7-badc-8aa2e9716071 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:12.240095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:12.240954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.241272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.241553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] PATH_INFO: `/v3/projects/ea8cd34bc01848e6bd12dcc4a2976d50` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.242947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8cd34bc01848e6bd12dcc4a2976d50)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:12.245192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:12.295388 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8cd34bc01848e6bd12dcc4a2976d50', 'request_id': 'req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:12.305695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8cd34bc01848e6bd12dcc4a2976d50', 'request_id': 'req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:12.315060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8cd34bc01848e6bd12dcc4a2976d50', 'request_id': 'req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:12.327863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] The token cache is being invalidate because project ea8cd34bc01848e6bd12dcc4a2976d50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:12.328148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:12.328867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7cdd16b1-e6bf-400c-81f6-a12c6f7d05ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:12.330495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1087/2168] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:12 2022] DELETE /identity/v3/projects/ea8cd34bc01848e6bd12dcc4a2976d50 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:12.335492 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379ed4-a02a-4a5e-a650-39cdf59f2251 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.335784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379ed4-a02a-4a5e-a650-39cdf59f2251 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.336272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c379ed4-a02a-4a5e-a650-39cdf59f2251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.355670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0c379ed4-a02a-4a5e-a650-39cdf59f2251 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:12.357282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c379ed4-a02a-4a5e-a650-39cdf59f2251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:12.417588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:12.427211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1082/2169] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:12.439308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f51c-bab3-4b51-84a8-591d9f40920a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.439621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f51c-bab3-4b51-84a8-591d9f40920a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.439952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9f51c-bab3-4b51-84a8-591d9f40920a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.462816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2ec9f51c-bab3-4b51-84a8-591d9f40920a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:12.464042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2ec9f51c-bab3-4b51-84a8-591d9f40920a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:12.472737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1088/2170] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:12.477737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c379ed4-a02a-4a5e-a650-39cdf59f2251 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:12.488757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c379ed4-a02a-4a5e-a650-39cdf59f2251 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:12.495084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:12.495769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.496050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.496322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.497087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:12.497460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:12.499080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:12.512036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1083/2171] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:24:12.517986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9f51c-bab3-4b51-84a8-591d9f40920a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:12.528507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec9f51c-bab3-4b51-84a8-591d9f40920a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:12.534605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a3c35-9c73-4b38-b910-efaa4c9ee908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:12.535266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a3c35-9c73-4b38-b910-efaa4c9ee908 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.535548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a3c35-9c73-4b38-b910-efaa4c9ee908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.535825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a3c35-9c73-4b38-b910-efaa4c9ee908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.536550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a3c35-9c73-4b38-b910-efaa4c9ee908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:12.538485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a3c35-9c73-4b38-b910-efaa4c9ee908 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:12.567883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1089/2172] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:12 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:12.573100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:12.586911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4bb723-74c5-4a82-81d3-8e4f1083ff37 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:12.647536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72205d6-21f1-487c-9786-fde14d9b6dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:12.648200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72205d6-21f1-487c-9786-fde14d9b6dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.649027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72205d6-21f1-487c-9786-fde14d9b6dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.649027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72205d6-21f1-487c-9786-fde14d9b6dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.649862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72205d6-21f1-487c-9786-fde14d9b6dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:12.651781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72205d6-21f1-487c-9786-fde14d9b6dd1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:12.672977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1084/2173] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:12 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:12.677379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a3c35-9c73-4b38-b910-efaa4c9ee908 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:12.688794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a3c35-9c73-4b38-b910-efaa4c9ee908 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:12.694263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad0278-324b-4ff8-8ffe-4cde1d311594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:12.694937 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad0278-324b-4ff8-8ffe-4cde1d311594 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.695222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad0278-324b-4ff8-8ffe-4cde1d311594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.695501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cad0278-324b-4ff8-8ffe-4cde1d311594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.696239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad0278-324b-4ff8-8ffe-4cde1d311594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:12.697886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cad0278-324b-4ff8-8ffe-4cde1d311594 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:12.709088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1090/2174] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:12.714039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72205d6-21f1-487c-9786-fde14d9b6dd1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:12.724988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72205d6-21f1-487c-9786-fde14d9b6dd1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:12.729925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed06c0-23d2-413e-8bb1-860dda9e872d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:12.730588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed06c0-23d2-413e-8bb1-860dda9e872d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.730869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed06c0-23d2-413e-8bb1-860dda9e872d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.731146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed06c0-23d2-413e-8bb1-860dda9e872d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.732188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed06c0-23d2-413e-8bb1-860dda9e872d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:12.733907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed06c0-23d2-413e-8bb1-860dda9e872d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:12.745805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1085/2175] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:12.750204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad0278-324b-4ff8-8ffe-4cde1d311594 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:12.762341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cad0278-324b-4ff8-8ffe-4cde1d311594 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:12.767872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:12.768803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.769086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.769396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] PATH_INFO: `/v3/projects/3621d57747c544a8aa4e683140cd9eaa/users/934d71e4968144ae9c0a885d2ab39fbc/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.786098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3621d57747c544a8aa4e683140cd9eaa, user_id=934d71e4968144ae9c0a885d2ab39fbc, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:12.788764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:12.803859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1091/2176] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:12 2022] PUT /identity/v3/projects/3621d57747c544a8aa4e683140cd9eaa/users/934d71e4968144ae9c0a885d2ab39fbc/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:12.814718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255473ca-28ac-4a1f-9998-5b4ccb024839 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:12.815014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255473ca-28ac-4a1f-9998-5b4ccb024839 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:12.815295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255473ca-28ac-4a1f-9998-5b4ccb024839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:12.843890 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-255473ca-28ac-4a1f-9998-5b4ccb024839 None None] MFA Rules not processed for user `934d71e4968144ae9c0a885d2ab39fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:12.845212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-255473ca-28ac-4a1f-9998-5b4ccb024839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:12.911721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1086/2177] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:12.923782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:12.925980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:12.933305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:12.998953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa84294-ba1d-4ac5-9c87-50d4675cdc5f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:13.004689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:13.005309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:13.006076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:13.006076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:13.017292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:13.017631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:13.027586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1092/2178] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:17.658246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255473ca-28ac-4a1f-9998-5b4ccb024839 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:17.660687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-255473ca-28ac-4a1f-9998-5b4ccb024839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:17.762684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:17.787326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255473ca-28ac-4a1f-9998-5b4ccb024839 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:17.787326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:17.787326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:17.787326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:17.787615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] PATH_INFO: `/v3/users/c8013b2b7a954f2d8fdd7e7d51ccf6f1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:17.787615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8013b2b7a954f2d8fdd7e7d51ccf6f1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:17.789011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:17.885265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8013b2b7a954f2d8fdd7e7d51ccf6f1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:17.886217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8013b2b7a954f2d8fdd7e7d51ccf6f1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:17.893425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8013b2b7a954f2d8fdd7e7d51ccf6f1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:17.897972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:17.898567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:17.921418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9fddb3-ad71-4973-962e-d3c2450e9d1d service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:17.926387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8013b2b7a954f2d8fdd7e7d51ccf6f1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:17.926776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:17.927763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:17.928116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:17.928466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] PATH_INFO: `/v3/users/b8c37bfdb2384b838f6e5a0c7b0c1b4d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:17.930082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c37bfdb2384b838f6e5a0c7b0c1b4d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:17.932184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:17.939138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e292fdc3-9156-4885-90e0-fc002a8b6d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8013b2b7a954f2d8fdd7e7d51ccf6f1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:17.977224 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1087/2179] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:17 2022] DELETE /identity/v3/users/c8013b2b7a954f2d8fdd7e7d51ccf6f1 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:17.979684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51729b-8cd5-4d7f-a4e5-5680b89d8383 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:17.980090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51729b-8cd5-4d7f-a4e5-5680b89d8383 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:17.980329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f51729b-8cd5-4d7f-a4e5-5680b89d8383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.003256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c37bfdb2384b838f6e5a0c7b0c1b4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.005523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4f51729b-8cd5-4d7f-a4e5-5680b89d8383 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:18.007981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f51729b-8cd5-4d7f-a4e5-5680b89d8383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.027842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c37bfdb2384b838f6e5a0c7b0c1b4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.028878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c37bfdb2384b838f6e5a0c7b0c1b4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.039938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c37bfdb2384b838f6e5a0c7b0c1b4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.049537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-199b7b1c-e1b2-4554-aa57-50180d6d26a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c37bfdb2384b838f6e5a0c7b0c1b4d'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.060578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1093/2180] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:17 2022] DELETE /identity/v3/users/b8c37bfdb2384b838f6e5a0c7b0c1b4d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:18.065390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0f9c-8bdd-41ec-a734-64928aa6b771 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.065726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0f9c-8bdd-41ec-a734-64928aa6b771 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.066180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c0f9c-8bdd-41ec-a734-64928aa6b771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.087950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7c3c0f9c-8bdd-41ec-a734-64928aa6b771 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:18.089381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7c3c0f9c-8bdd-41ec-a734-64928aa6b771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.115659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1088/2181] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:18.128855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51729b-8cd5-4d7f-a4e5-5680b89d8383 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:18.130402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f51729b-8cd5-4d7f-a4e5-5680b89d8383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.172997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1094/2182] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:18.179016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c0f9c-8bdd-41ec-a734-64928aa6b771 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:18.181050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7c3c0f9c-8bdd-41ec-a734-64928aa6b771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.204692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:18.206624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f51729b-8cd5-4d7f-a4e5-5680b89d8383 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:18.241346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:18.246099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c0f9c-8bdd-41ec-a734-64928aa6b771 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:18.250160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:18.250855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.251462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.251462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.263091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:18.264541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:18.264977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:18.265692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.265982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.266268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.275088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1095/2183] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:24:18.276727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:18.278257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:18.290480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1089/2184] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:24:18.504030 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:18.505721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.526605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd55c5-9c22-48e5-a753-2bc7716a17bb service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:18.532379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:18.533330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.533501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.533864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] PATH_INFO: `/v3/users/a034fb8b4b214bf29e8d5cc88f62d687` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.535233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a034fb8b4b214bf29e8d5cc88f62d687)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:18.537315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:18.615292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a034fb8b4b214bf29e8d5cc88f62d687'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.624364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:18.628319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.657901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a034fb8b4b214bf29e8d5cc88f62d687'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.658718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456cc8e-c061-4654-9462-62f58b568ba2 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:18.659313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a034fb8b4b214bf29e8d5cc88f62d687'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.665780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:18.666560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.666731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.666998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] PATH_INFO: `/v3/projects/1bc238aba75b405a9e073025c1e1fb85` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.668274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc238aba75b405a9e073025c1e1fb85)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:18.670468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:18.676705 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a034fb8b4b214bf29e8d5cc88f62d687'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.687174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a034fb8b4b214bf29e8d5cc88f62d687'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.693946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1096/2185] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:18 2022] DELETE /identity/v3/users/a034fb8b4b214bf29e8d5cc88f62d687 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:18.694838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:18.696650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.726411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc238aba75b405a9e073025c1e1fb85', 'request_id': 'req-573f6a1f-ba4a-4bda-bf91-383999974298'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.733562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc238aba75b405a9e073025c1e1fb85', 'request_id': 'req-573f6a1f-ba4a-4bda-bf91-383999974298'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.741623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc238aba75b405a9e073025c1e1fb85', 'request_id': 'req-573f6a1f-ba4a-4bda-bf91-383999974298'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.754655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] The token cache is being invalidate because project 1bc238aba75b405a9e073025c1e1fb85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:18.755027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.755724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-573f6a1f-ba4a-4bda-bf91-383999974298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.757360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1090/2186] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:18 2022] DELETE /identity/v3/projects/1bc238aba75b405a9e073025c1e1fb85 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:18.760954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bece-4557-48cb-81c0-543c278be313 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.760954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bece-4557-48cb-81c0-543c278be313 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.760954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6395bece-4557-48cb-81c0-543c278be313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.774077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdec35-c15f-458d-8467-832ccf2b3316 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:18.781472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6395bece-4557-48cb-81c0-543c278be313 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:18.782800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6395bece-4557-48cb-81c0-543c278be313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.818325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:18.841597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:18.842604 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.842997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.843394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] PATH_INFO: `/v3/projects/489df5c12d274a23be1c0bf87dc09dc7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.844790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=489df5c12d274a23be1c0bf87dc09dc7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:18.846834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:18.850347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1091/2187] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:18.851680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051073d-5901-4cb4-85ee-6c466146a819 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.851995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051073d-5901-4cb4-85ee-6c466146a819 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.852424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051073d-5901-4cb4-85ee-6c466146a819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.883355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7051073d-5901-4cb4-85ee-6c466146a819 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:18.885083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7051073d-5901-4cb4-85ee-6c466146a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.905350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '489df5c12d274a23be1c0bf87dc09dc7', 'request_id': 'req-23e94025-9cbf-4503-9474-4f3162e4d810'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.923304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '489df5c12d274a23be1c0bf87dc09dc7', 'request_id': 'req-23e94025-9cbf-4503-9474-4f3162e4d810'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.931169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '489df5c12d274a23be1c0bf87dc09dc7', 'request_id': 'req-23e94025-9cbf-4503-9474-4f3162e4d810'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.944100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] The token cache is being invalidate because project 489df5c12d274a23be1c0bf87dc09dc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:18.944389 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.945112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:18.946814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1097/2188] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:18 2022] DELETE /identity/v3/projects/489df5c12d274a23be1c0bf87dc09dc7 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:18.947841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:18.949237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:18.956603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:18.973366 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1092/2189] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:18.976245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba90dc-8764-4b7e-9ce2-6cee1d8dfb45 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:18.976966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba90dc-8764-4b7e-9ce2-6cee1d8dfb45 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:18.977450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cba90dc-8764-4b7e-9ce2-6cee1d8dfb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:18.997529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1cba90dc-8764-4b7e-9ce2-6cee1d8dfb45 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:18.998732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1cba90dc-8764-4b7e-9ce2-6cee1d8dfb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.008066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1093/2190] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:19.015665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6693cac-5f86-4a1e-a9b2-cf5a5a59351b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.016137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6693cac-5f86-4a1e-a9b2-cf5a5a59351b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.016423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6693cac-5f86-4a1e-a9b2-cf5a5a59351b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.017943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e94025-9cbf-4503-9474-4f3162e4d810 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.021561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.022218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.022547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.023277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.025869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.037206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b6693cac-5f86-4a1e-a9b2-cf5a5a59351b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:19.038530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.038887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b6693cac-5f86-4a1e-a9b2-cf5a5a59351b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.040037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.049247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1094/2191] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:19.050308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1098/2192] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:19.050869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0efe8f-752a-4274-8b31-a79d2b6cbb72 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.051296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0efe8f-752a-4274-8b31-a79d2b6cbb72 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.051682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0efe8f-752a-4274-8b31-a79d2b6cbb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.052770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.063263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fe929f-8f41-44f2-a14d-790229df364b service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.068425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.069102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.069431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.069712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.070722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:19.071398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.073056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.078050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-df0efe8f-752a-4274-8b31-a79d2b6cbb72 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:19.079522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-df0efe8f-752a-4274-8b31-a79d2b6cbb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.088499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1095/2193] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:19.089788 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1099/2194] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:24:19.092971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0efe8f-752a-4274-8b31-a79d2b6cbb72 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.094408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.104182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0efe8f-752a-4274-8b31-a79d2b6cbb72 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.112875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965355c-32e6-4001-9368-82281a98cea9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.113193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.113193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.113193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.113193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.114408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:19.114750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.116601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.123637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9cee9-60ba-4362-95f5-a57795d16a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.124798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9cee9-60ba-4362-95f5-a57795d16a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.125888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9cee9-60ba-4362-95f5-a57795d16a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.125888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba9cee9-60ba-4362-95f5-a57795d16a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.126586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9cee9-60ba-4362-95f5-a57795d16a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.129652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba9cee9-60ba-4362-95f5-a57795d16a3b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.142137 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1096/2195] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:19.143279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.154847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7c00b-4cd9-4386-8e30-cce4fe2d109a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.162299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646d834-bf6f-4821-a497-68fe635cd763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.163301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646d834-bf6f-4821-a497-68fe635cd763 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.163592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646d834-bf6f-4821-a497-68fe635cd763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.163932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646d834-bf6f-4821-a497-68fe635cd763 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.164776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646d834-bf6f-4821-a497-68fe635cd763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.167086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646d834-bf6f-4821-a497-68fe635cd763 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.169878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1100/2196] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/projects => generated 391 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:19.174357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9cee9-60ba-4362-95f5-a57795d16a3b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.217103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1097/2197] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/projects => generated 375 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:19.219286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646d834-bf6f-4821-a497-68fe635cd763 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.242351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:19.273351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba9cee9-60ba-4362-95f5-a57795d16a3b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.310669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646d834-bf6f-4821-a497-68fe635cd763 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.315885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.316575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.316861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.317221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.317870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.319924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.343820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1098/2198] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:19.345092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.347151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.356208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c338226-8956-4ec9-8273-f47fe2d2e83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.357399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c338226-8956-4ec9-8273-f47fe2d2e83d admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.357652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c338226-8956-4ec9-8273-f47fe2d2e83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.363741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c338226-8956-4ec9-8273-f47fe2d2e83d admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.364657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c338226-8956-4ec9-8273-f47fe2d2e83d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.368291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757a569a-22ef-40b2-85c7-13aa8dbec59a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.371039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c338226-8956-4ec9-8273-f47fe2d2e83d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.376920 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.377887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.378176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.378455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] PATH_INFO: `/v3/users/760eb98fa66947378c33f79a856e22f7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.380500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=760eb98fa66947378c33f79a856e22f7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.383088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.399289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1101/2199] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/users => generated 296 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:19.402140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c338226-8956-4ec9-8273-f47fe2d2e83d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.416268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c338226-8956-4ec9-8273-f47fe2d2e83d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.422128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24757a81-ead0-472e-a854-e4abfaedf830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.422967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24757a81-ead0-472e-a854-e4abfaedf830 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.423255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24757a81-ead0-472e-a854-e4abfaedf830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.423533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24757a81-ead0-472e-a854-e4abfaedf830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.424446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24757a81-ead0-472e-a854-e4abfaedf830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.426101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24757a81-ead0-472e-a854-e4abfaedf830 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.438758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1102/2200] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:19.440246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24757a81-ead0-472e-a854-e4abfaedf830 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.451601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24757a81-ead0-472e-a854-e4abfaedf830 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.452462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '760eb98fa66947378c33f79a856e22f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:19.453289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '760eb98fa66947378c33f79a856e22f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:19.453868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '760eb98fa66947378c33f79a856e22f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:19.457693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.458486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.458836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.459193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.460158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.461664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.470600 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '760eb98fa66947378c33f79a856e22f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:19.480270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1103/2201] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:19.484575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.486641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '760eb98fa66947378c33f79a856e22f7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:19.492935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.512873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d75aa2-5055-4565-b3fd-fd1463386090 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.518455 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96fb47-0080-42ec-8687-0a04fa14a2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.519150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96fb47-0080-42ec-8687-0a04fa14a2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.519484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96fb47-0080-42ec-8687-0a04fa14a2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.519768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96fb47-0080-42ec-8687-0a04fa14a2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.520512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96fb47-0080-42ec-8687-0a04fa14a2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.522889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96fb47-0080-42ec-8687-0a04fa14a2f9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.526357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1099/2202] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:19 2022] DELETE /identity/v3/users/760eb98fa66947378c33f79a856e22f7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:19.529415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.531291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.539850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:19.546999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1104/2203] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:19.553040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ae365-86f9-4713-9bf3-7a4dc02d802d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.553386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ae365-86f9-4713-9bf3-7a4dc02d802d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.553687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ae365-86f9-4713-9bf3-7a4dc02d802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.575567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6b3ae365-86f9-4713-9bf3-7a4dc02d802d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:19.577284 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6b3ae365-86f9-4713-9bf3-7a4dc02d802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.620245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e4dd2-45b1-40c0-b8b5-08a34a204bef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.626497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dd8d2-06d4-4f0b-9e9f-4a435e701a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.627423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dd8d2-06d4-4f0b-9e9f-4a435e701a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.627954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dd8d2-06d4-4f0b-9e9f-4a435e701a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.628449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266dd8d2-06d4-4f0b-9e9f-4a435e701a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.629504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dd8d2-06d4-4f0b-9e9f-4a435e701a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.631450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266dd8d2-06d4-4f0b-9e9f-4a435e701a0f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.645745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1100/2204] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:19.646835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dd8d2-06d4-4f0b-9e9f-4a435e701a0f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.665447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266dd8d2-06d4-4f0b-9e9f-4a435e701a0f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.677471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.677471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.677727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.678002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] PATH_INFO: `/v3/projects/0d6d5610e5a240eeb78ae85ed576a672/users/bdc51ac235e14df3bab3bd3e4fefc669/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.678907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1105/2205] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:19.680965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ae365-86f9-4713-9bf3-7a4dc02d802d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.693959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ae365-86f9-4713-9bf3-7a4dc02d802d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.694540 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6d5610e5a240eeb78ae85ed576a672, user_id=bdc51ac235e14df3bab3bd3e4fefc669, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.697682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.702135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b537b-7c92-4512-b6d8-9196eb2a3f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.703412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b537b-7c92-4512-b6d8-9196eb2a3f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.703762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b537b-7c92-4512-b6d8-9196eb2a3f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.704009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b537b-7c92-4512-b6d8-9196eb2a3f74 admin admin] PATH_INFO: `/v3/projects/2725917e195e47fbb93e5d9e7b1e8802/users/a449ba1022f345fea6484e5ae173b3e2/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.719144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1101/2206] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:19 2022] PUT /identity/v3/projects/0d6d5610e5a240eeb78ae85ed576a672/users/bdc51ac235e14df3bab3bd3e4fefc669/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:19.720428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:19.730969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.735052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b537b-7c92-4512-b6d8-9196eb2a3f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2725917e195e47fbb93e5d9e7b1e8802, user_id=a449ba1022f345fea6484e5ae173b3e2, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:19.738148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b537b-7c92-4512-b6d8-9196eb2a3f74 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:19.759198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1106/2207] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:19 2022] PUT /identity/v3/projects/2725917e195e47fbb93e5d9e7b1e8802/users/a449ba1022f345fea6484e5ae173b3e2/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:19.760856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33c64b-cbbf-4465-b048-338ad69d39f6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.761151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33c64b-cbbf-4465-b048-338ad69d39f6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.761536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e33c64b-cbbf-4465-b048-338ad69d39f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.800368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3e33c64b-cbbf-4465-b048-338ad69d39f6 None None] MFA Rules not processed for user `bdc51ac235e14df3bab3bd3e4fefc669`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:19.801660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3e33c64b-cbbf-4465-b048-338ad69d39f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:19.813993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:19.844324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac169b66-116f-4fa7-a372-336e20da5de9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:19.907820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1107/2208] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:19.914142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd4093-13c8-4546-a098-ffc7656ef594 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.914703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd4093-13c8-4546-a098-ffc7656ef594 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.915053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fd4093-13c8-4546-a098-ffc7656ef594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.943651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:19.944337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:19.944631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:19.944891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:19.954638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-09fd4093-13c8-4546-a098-ffc7656ef594 None None] MFA Rules not processed for user `a449ba1022f345fea6484e5ae173b3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:19.956299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-09fd4093-13c8-4546-a098-ffc7656ef594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.022485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:20.048233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.050046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.063008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1102/2209] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:20.063886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:20.065374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.070024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1108/2210] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:20.071602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd4093-13c8-4546-a098-ffc7656ef594 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:20.076989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-09fd4093-13c8-4546-a098-ffc7656ef594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.080237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6724c62-bb78-4278-a985-bb5cbf8d7355 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:20.087086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:20.088469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.088750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.089026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] PATH_INFO: `/v3/projects/c2d34586f25847bf902d8bdc494b89da` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.090359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d34586f25847bf902d8bdc494b89da)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.093214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.148011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d34586f25847bf902d8bdc494b89da', 'request_id': 'req-d44aa998-93b0-4bee-b3b7-f335edbdd260'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.155274 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d34586f25847bf902d8bdc494b89da', 'request_id': 'req-d44aa998-93b0-4bee-b3b7-f335edbdd260'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.165683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d34586f25847bf902d8bdc494b89da', 'request_id': 'req-d44aa998-93b0-4bee-b3b7-f335edbdd260'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.188021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] The token cache is being invalidate because project c2d34586f25847bf902d8bdc494b89da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:20.188387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.189588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.191463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1103/2211] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:20 2022] DELETE /identity/v3/projects/c2d34586f25847bf902d8bdc494b89da => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:20.192147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:20.194090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.202925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fd4093-13c8-4546-a098-ffc7656ef594 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:20.248524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:20.288689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44aa998-93b0-4bee-b3b7-f335edbdd260 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:20.294175 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb98aa0-4423-4fae-b91d-a7aa63d9a553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:20.294976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb98aa0-4423-4fae-b91d-a7aa63d9a553 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.295420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb98aa0-4423-4fae-b91d-a7aa63d9a553 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.295824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb98aa0-4423-4fae-b91d-a7aa63d9a553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.297601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cdb98aa0-4423-4fae-b91d-a7aa63d9a553 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.311353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae25745-eddf-4829-9971-735419561200 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:20.312136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae25745-eddf-4829-9971-735419561200 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.312429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae25745-eddf-4829-9971-735419561200 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.312783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae25745-eddf-4829-9971-735419561200 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.315084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-aae25745-eddf-4829-9971-735419561200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.377022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb98aa0-4423-4fae-b91d-a7aa63d9a553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.378530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb98aa0-4423-4fae-b91d-a7aa63d9a553 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.385391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae25745-eddf-4829-9971-735419561200 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.387021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae25745-eddf-4829-9971-735419561200 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.389014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1104/2212] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:20.397851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0afe-a90b-4587-9b8d-5418198b97bc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.399038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0afe-a90b-4587-9b8d-5418198b97bc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.399370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0afe-a90b-4587-9b8d-5418198b97bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.401593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1109/2213] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:20.418589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae25745-eddf-4829-9971-735419561200 service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:20.420446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-aae25745-eddf-4829-9971-735419561200 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.425622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d84b0afe-a90b-4587-9b8d-5418198b97bc None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:20.425622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d84b0afe-a90b-4587-9b8d-5418198b97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.451867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:20.524773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae25745-eddf-4829-9971-735419561200 service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:20.529279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1105/2214] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:20.529987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:20.530553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.530839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.531119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] PATH_INFO: `/v3/projects/79128a84c79e4690bec90d0bfc838e09` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.532335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79128a84c79e4690bec90d0bfc838e09)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.534290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.538172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d79c61-fcea-4594-afe2-ee07dbcc0eb3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.538589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d79c61-fcea-4594-afe2-ee07dbcc0eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.539094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d79c61-fcea-4594-afe2-ee07dbcc0eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.575573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b6d79c61-fcea-4594-afe2-ee07dbcc0eb3 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:20.578477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b6d79c61-fcea-4594-afe2-ee07dbcc0eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.591403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1106/2215] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:20.596509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d79c61-fcea-4594-afe2-ee07dbcc0eb3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:20.618973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d79c61-fcea-4594-afe2-ee07dbcc0eb3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:20.630822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:20.631846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.632134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.634009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.634809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:20.635130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.636765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.642086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79128a84c79e4690bec90d0bfc838e09', 'request_id': 'req-0d336a18-8df5-4536-91fa-122d9243e926'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.652085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1107/2216] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:20.656155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79128a84c79e4690bec90d0bfc838e09', 'request_id': 'req-0d336a18-8df5-4536-91fa-122d9243e926'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.656294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:20.668615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d87516-7149-45ae-8dad-731dd4b5e548 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:20.669821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79128a84c79e4690bec90d0bfc838e09', 'request_id': 'req-0d336a18-8df5-4536-91fa-122d9243e926'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.703687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] The token cache is being invalidate because project 79128a84c79e4690bec90d0bfc838e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:20.704003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.714501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.714501 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1110/2217] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:20 2022] DELETE /identity/v3/projects/79128a84c79e4690bec90d0bfc838e09 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:24:20.723415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:20.725283 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.744859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:20.767116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:20.768183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.768760 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.769303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.770465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.773430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.814218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1108/2218] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:20 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:20.815262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:20.817155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:20.834788 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d336a18-8df5-4536-91fa-122d9243e926 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:20.920021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6d322-781e-479a-bba3-83a7946416b5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:20.926240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:20.927233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.927845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.928244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] PATH_INFO: `/v3/users/bdc51ac235e14df3bab3bd3e4fefc669` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.930189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc51ac235e14df3bab3bd3e4fefc669)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.933269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.934972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:20.936058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:20.936341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:20.937215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] PATH_INFO: `/v3/users/2f9a34806fb042518691211815916ccb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:20.938540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9a34806fb042518691211815916ccb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:20.940610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:20.973625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:20.996225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc51ac235e14df3bab3bd3e4fefc669'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.996871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc51ac235e14df3bab3bd3e4fefc669'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.997270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9a34806fb042518691211815916ccb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:20.997712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc51ac235e14df3bab3bd3e4fefc669'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.012221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc51ac235e14df3bab3bd3e4fefc669'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.013238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9a34806fb042518691211815916ccb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.014696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9a34806fb042518691211815916ccb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.022602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8af4572a-1efa-4c33-b2f1-3329cae12760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc51ac235e14df3bab3bd3e4fefc669'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.025943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9a34806fb042518691211815916ccb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.035399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9a34806fb042518691211815916ccb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.042261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1111/2219] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:20 2022] DELETE /identity/v3/users/2f9a34806fb042518691211815916ccb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:21.043037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.044367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.050508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1109/2220] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:20 2022] DELETE /identity/v3/users/bdc51ac235e14df3bab3bd3e4fefc669 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:21.055549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906e9f7-4213-4ae8-976d-3e271795b333 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.055987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906e9f7-4213-4ae8-976d-3e271795b333 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.056307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906e9f7-4213-4ae8-976d-3e271795b333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.091817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c906e9f7-4213-4ae8-976d-3e271795b333 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:21.092542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:21.095136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c906e9f7-4213-4ae8-976d-3e271795b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.148372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07919b-8b75-4f17-acbc-6f8a16022273 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.210389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1110/2221] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:21.226459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906e9f7-4213-4ae8-976d-3e271795b333 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.228114 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c906e9f7-4213-4ae8-976d-3e271795b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.282288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658fc83-72e4-4bf2-ad34-ad02fd477366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.282977 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658fc83-72e4-4bf2-ad34-ad02fd477366 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.283257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658fc83-72e4-4bf2-ad34-ad02fd477366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.283529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658fc83-72e4-4bf2-ad34-ad02fd477366 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.284239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658fc83-72e4-4bf2-ad34-ad02fd477366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.286368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658fc83-72e4-4bf2-ad34-ad02fd477366 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.316340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1112/2222] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:21 2022] POST /identity/v3/users => generated 288 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:21.320776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658fc83-72e4-4bf2-ad34-ad02fd477366 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.332770 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658fc83-72e4-4bf2-ad34-ad02fd477366 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.334637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906e9f7-4213-4ae8-976d-3e271795b333 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.339045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297d0ca-a04b-4a70-80af-37b2564444dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.340256 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297d0ca-a04b-4a70-80af-37b2564444dd admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.340605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297d0ca-a04b-4a70-80af-37b2564444dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.340953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297d0ca-a04b-4a70-80af-37b2564444dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.341837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297d0ca-a04b-4a70-80af-37b2564444dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.343494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801482-993a-4852-bd10-82395c1e454a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.344408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297d0ca-a04b-4a70-80af-37b2564444dd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.345423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801482-993a-4852-bd10-82395c1e454a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.346848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801482-993a-4852-bd10-82395c1e454a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.347431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54801482-993a-4852-bd10-82395c1e454a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.364072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54801482-993a-4852-bd10-82395c1e454a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.364072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54801482-993a-4852-bd10-82395c1e454a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.365138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1113/2223] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:21.366179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297d0ca-a04b-4a70-80af-37b2564444dd admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.378175 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1111/2224] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:21.379545 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297d0ca-a04b-4a70-80af-37b2564444dd admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.385834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.386526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.386952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.387265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.388120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.389881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.406296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1114/2225] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:21.408257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801482-993a-4852-bd10-82395c1e454a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.435934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54801482-993a-4852-bd10-82395c1e454a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.447795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8743bc-6566-4d75-9fdc-3970ad616a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.448710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8743bc-6566-4d75-9fdc-3970ad616a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.448999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8743bc-6566-4d75-9fdc-3970ad616a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.449316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8743bc-6566-4d75-9fdc-3970ad616a8a admin admin] PATH_INFO: `/v3/projects/9d89beafb10246f7b9087ca7cd75f239/users/da4b12d5d8f647589a2226303b01c2f3/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.483119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8743bc-6566-4d75-9fdc-3970ad616a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d89beafb10246f7b9087ca7cd75f239, user_id=da4b12d5d8f647589a2226303b01c2f3, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.489269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8743bc-6566-4d75-9fdc-3970ad616a8a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.496352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.498255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.506919 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1112/2226] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:21 2022] PUT /identity/v3/projects/9d89beafb10246f7b9087ca7cd75f239/users/da4b12d5d8f647589a2226303b01c2f3/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:21.517897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1855f9-7295-4695-9fef-19ad4aaa6d0d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.518194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1855f9-7295-4695-9fef-19ad4aaa6d0d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.518548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1855f9-7295-4695-9fef-19ad4aaa6d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.522229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47080cd-cd5e-4327-8755-a5ab432aee24 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.550845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4b1855f9-7295-4695-9fef-19ad4aaa6d0d None None] MFA Rules not processed for user `da4b12d5d8f647589a2226303b01c2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:21.552151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b1855f9-7295-4695-9fef-19ad4aaa6d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.567225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:21.628653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.629879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.630201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.630440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] PATH_INFO: `/v3/projects/327b87f2c7fb4fa5b5e6e34b48a1bf38` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.631801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327b87f2c7fb4fa5b5e6e34b48a1bf38)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.634123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.648310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1113/2227] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:21.671683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1855f9-7295-4695-9fef-19ad4aaa6d0d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.673604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b1855f9-7295-4695-9fef-19ad4aaa6d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.694592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:21.696760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327b87f2c7fb4fa5b5e6e34b48a1bf38', 'request_id': 'req-18997bdc-8251-4af7-b25d-8fe12eeacaf1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.708033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327b87f2c7fb4fa5b5e6e34b48a1bf38', 'request_id': 'req-18997bdc-8251-4af7-b25d-8fe12eeacaf1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.715439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327b87f2c7fb4fa5b5e6e34b48a1bf38', 'request_id': 'req-18997bdc-8251-4af7-b25d-8fe12eeacaf1'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.727895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] The token cache is being invalidate because project 327b87f2c7fb4fa5b5e6e34b48a1bf38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:21.728369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.729047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.730948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1115/2228] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:21 2022] DELETE /identity/v3/projects/327b87f2c7fb4fa5b5e6e34b48a1bf38 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:21.736202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.737631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.743567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1855f9-7295-4695-9fef-19ad4aaa6d0d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.809282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.810489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.811078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.811553 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.814601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.823092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18997bdc-8251-4af7-b25d-8fe12eeacaf1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.830876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.831913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.832213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.832510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] PATH_INFO: `/v3/users/c6a2b185755d4472830abeaab149e44f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.834404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a2b185755d4472830abeaab149e44f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.837989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.892435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.893967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.903222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a2b185755d4472830abeaab149e44f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.904592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1114/2229] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:21.905295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:21.906801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:21.924427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65df9d0-fd86-40ef-bfa5-a51c480f3d57 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:21.930565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:21.931342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:21.931630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:21.931827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a2b185755d4472830abeaab149e44f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.931932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] PATH_INFO: `/v3/projects/0d6d5610e5a240eeb78ae85ed576a672` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:21.933139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a2b185755d4472830abeaab149e44f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.934293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6d5610e5a240eeb78ae85ed576a672)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:21.938451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:21.943669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a2b185755d4472830abeaab149e44f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.956367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-d8239d2c-696c-4434-932b-9fb1e60b48f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a2b185755d4472830abeaab149e44f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:21.964169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:21.968743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1116/2230] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:21 2022] DELETE /identity/v3/users/c6a2b185755d4472830abeaab149e44f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:22.029300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6d5610e5a240eeb78ae85ed576a672', 'request_id': 'req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.052585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6d5610e5a240eeb78ae85ed576a672', 'request_id': 'req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.069002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6d5610e5a240eeb78ae85ed576a672', 'request_id': 'req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.101683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] The token cache is being invalidate because project 0d6d5610e5a240eeb78ae85ed576a672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:22.102140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.102911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cc58dc45-ee79-46a0-8891-4863d9d1bb7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.109781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1115/2231] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:21 2022] DELETE /identity/v3/projects/0d6d5610e5a240eeb78ae85ed576a672 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:22.111513 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc6a77-799e-4b95-bb5e-4aed09a7b286 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.112316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc6a77-799e-4b95-bb5e-4aed09a7b286 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.112316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc6a77-799e-4b95-bb5e-4aed09a7b286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.136436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0ccc6a77-799e-4b95-bb5e-4aed09a7b286 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:22.137850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0ccc6a77-799e-4b95-bb5e-4aed09a7b286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:22.212978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:22.257252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1117/2232] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:22.263573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60280-2f3c-4e6b-888b-cb3878c34369 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.263870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60280-2f3c-4e6b-888b-cb3878c34369 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.264211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e60280-2f3c-4e6b-888b-cb3878c34369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.300381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-52e60280-2f3c-4e6b-888b-cb3878c34369 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:22.301669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52e60280-2f3c-4e6b-888b-cb3878c34369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:22.311144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1116/2233] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:22.315407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc6a77-799e-4b95-bb5e-4aed09a7b286 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.327721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc6a77-799e-4b95-bb5e-4aed09a7b286 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.337523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.338136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e60280-2f3c-4e6b-888b-cb3878c34369 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.339016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.339308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.339587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.340342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:22.340704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.341393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-52e60280-2f3c-4e6b-888b-cb3878c34369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:22.342638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.360472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1118/2234] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:24:22.365200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.379844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e60280-2f3c-4e6b-888b-cb3878c34369 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.380721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf24aa-ff19-4075-8a38-e47e3c15f7c2 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.387201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24768660-6c8b-422a-aa41-1f4ff33ba287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.387997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24768660-6c8b-422a-aa41-1f4ff33ba287 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.388455 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24768660-6c8b-422a-aa41-1f4ff33ba287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.388789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24768660-6c8b-422a-aa41-1f4ff33ba287 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.389597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24768660-6c8b-422a-aa41-1f4ff33ba287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.390951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.392887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24768660-6c8b-422a-aa41-1f4ff33ba287 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.409487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.409487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.409487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] PATH_INFO: `/v3/projects/5009ae5345d442db98c791c7989bf38a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.411699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5009ae5345d442db98c791c7989bf38a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.414973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.428794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1119/2235] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:22 2022] POST /identity/v3/projects => generated 403 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:22.432773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24768660-6c8b-422a-aa41-1f4ff33ba287 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.528585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24768660-6c8b-422a-aa41-1f4ff33ba287 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.557074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5009ae5345d442db98c791c7989bf38a', 'request_id': 'req-9860a405-a973-47c0-a863-8743c12027ba'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.567088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5009ae5345d442db98c791c7989bf38a', 'request_id': 'req-9860a405-a973-47c0-a863-8743c12027ba'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.578792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5009ae5345d442db98c791c7989bf38a', 'request_id': 'req-9860a405-a973-47c0-a863-8743c12027ba'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.598714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] The token cache is being invalidate because project 5009ae5345d442db98c791c7989bf38a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:22.599205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.600253 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:22.602307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1117/2236] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:22 2022] DELETE /identity/v3/projects/5009ae5345d442db98c791c7989bf38a => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:24:22.609315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41102c-e78f-4eb7-8666-965969d5ec0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.610033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41102c-e78f-4eb7-8666-965969d5ec0e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.610325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41102c-e78f-4eb7-8666-965969d5ec0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.610610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41102c-e78f-4eb7-8666-965969d5ec0e admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.611327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41102c-e78f-4eb7-8666-965969d5ec0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.613484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e41102c-e78f-4eb7-8666-965969d5ec0e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.637567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1120/2237] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:22 2022] POST /identity/v3/users => generated 302 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:22.641893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.643584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:22.658946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860a405-a973-47c0-a863-8743c12027ba admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.664615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aae076-dd44-4e44-8254-2137b7db324a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.665341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aae076-dd44-4e44-8254-2137b7db324a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.665634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aae076-dd44-4e44-8254-2137b7db324a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.665915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aae076-dd44-4e44-8254-2137b7db324a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.666698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aae076-dd44-4e44-8254-2137b7db324a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.668469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aae076-dd44-4e44-8254-2137b7db324a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.684306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1118/2238] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:22.685688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41102c-e78f-4eb7-8666-965969d5ec0e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.696943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e41102c-e78f-4eb7-8666-965969d5ec0e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.702158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d122371-66df-4149-a4e1-d4efb911d213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.702886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d122371-66df-4149-a4e1-d4efb911d213 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.703169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d122371-66df-4149-a4e1-d4efb911d213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.703447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d122371-66df-4149-a4e1-d4efb911d213 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.704197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d122371-66df-4149-a4e1-d4efb911d213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.705877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d122371-66df-4149-a4e1-d4efb911d213 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.717890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1121/2239] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:24:22.722520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aae076-dd44-4e44-8254-2137b7db324a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.735189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aae076-dd44-4e44-8254-2137b7db324a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.740907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.741868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.742159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.742439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] PATH_INFO: `/v3/projects/3058799973fc4d19a77910e259126579/users/9b4bc4411ea9473080f88cf429869511/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.760836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3058799973fc4d19a77910e259126579, user_id=9b4bc4411ea9473080f88cf429869511, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.763575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.779909 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1119/2240] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:22 2022] PUT /identity/v3/projects/3058799973fc4d19a77910e259126579/users/9b4bc4411ea9473080f88cf429869511/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:22.785139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8e112-a5f2-4fc6-a2a7-846240a3795f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.785531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8e112-a5f2-4fc6-a2a7-846240a3795f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.785882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8e112-a5f2-4fc6-a2a7-846240a3795f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.807638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1ac8e112-a5f2-4fc6-a2a7-846240a3795f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:22.808878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1ac8e112-a5f2-4fc6-a2a7-846240a3795f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:22.857463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:22.877441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1122/2241] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:22.888300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:22.890139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:22.949888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:22.954688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60864647-5c2c-4e86-8054-3a1167f2b225 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:22.960293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c60f51-752b-4a6e-a9f4-94003f77ac96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:22.961083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c60f51-752b-4a6e-a9f4-94003f77ac96 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:22.961808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c60f51-752b-4a6e-a9f4-94003f77ac96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:22.962297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c60f51-752b-4a6e-a9f4-94003f77ac96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:22.972860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c60f51-752b-4a6e-a9f4-94003f77ac96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:22.974280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c60f51-752b-4a6e-a9f4-94003f77ac96 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:22.984941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1120/2242] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:23.669877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571d848-abfb-4440-a0a3-b404b0cd73b9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.670437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571d848-abfb-4440-a0a3-b404b0cd73b9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:23.670526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571d848-abfb-4440-a0a3-b404b0cd73b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:23.698183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7571d848-abfb-4440-a0a3-b404b0cd73b9 None None] MFA Rules not processed for user `9b4bc4411ea9473080f88cf429869511`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:23.699420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7571d848-abfb-4440-a0a3-b404b0cd73b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:23.777160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1123/2243] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:23.781846 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c60f51-752b-4a6e-a9f4-94003f77ac96 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:23.792059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c60f51-752b-4a6e-a9f4-94003f77ac96 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:23.796772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a6970-8db7-416a-9462-e40fcc500dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:23.797476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a6970-8db7-416a-9462-e40fcc500dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.798046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a6970-8db7-416a-9462-e40fcc500dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:23.798046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a6970-8db7-416a-9462-e40fcc500dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:23.798727 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a6970-8db7-416a-9462-e40fcc500dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:23.800633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a6970-8db7-416a-9462-e40fcc500dd8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:23.828460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1121/2244] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:23 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:23.833093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571d848-abfb-4440-a0a3-b404b0cd73b9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:23.844496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571d848-abfb-4440-a0a3-b404b0cd73b9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:23.912302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02242c1-5a70-4e48-90a5-0a501c9f1c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:23.913149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02242c1-5a70-4e48-90a5-0a501c9f1c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.913483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02242c1-5a70-4e48-90a5-0a501c9f1c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:23.913747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02242c1-5a70-4e48-90a5-0a501c9f1c51 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:23.914614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02242c1-5a70-4e48-90a5-0a501c9f1c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:23.916555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02242c1-5a70-4e48-90a5-0a501c9f1c51 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:23.937908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1124/2245] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:23 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:23.942821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a6970-8db7-416a-9462-e40fcc500dd8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:23.953667 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a6970-8db7-416a-9462-e40fcc500dd8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:23.958315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef03ff-9f64-4ed7-8f16-1d4e4dcd6e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:23.958994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef03ff-9f64-4ed7-8f16-1d4e4dcd6e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.959275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef03ff-9f64-4ed7-8f16-1d4e4dcd6e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:23.959553 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef03ff-9f64-4ed7-8f16-1d4e4dcd6e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:23.960283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef03ff-9f64-4ed7-8f16-1d4e4dcd6e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:23.962228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdef03ff-9f64-4ed7-8f16-1d4e4dcd6e32 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:23.973802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1122/2246] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:23.979025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02242c1-5a70-4e48-90a5-0a501c9f1c51 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:23.990272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02242c1-5a70-4e48-90a5-0a501c9f1c51 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:23.997109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce8ab-2ced-4169-b201-3857b105a30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:23.998191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce8ab-2ced-4169-b201-3857b105a30c admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:23.998308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce8ab-2ced-4169-b201-3857b105a30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:24.001741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce8ab-2ced-4169-b201-3857b105a30c admin admin] PATH_INFO: `/v3/projects/f99b08d1a91f435c886dc463a9cfa85d/users/5e15b5b7f803454498a863bf3bac52b4/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:24.016646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dce8ab-2ced-4169-b201-3857b105a30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99b08d1a91f435c886dc463a9cfa85d, user_id=5e15b5b7f803454498a863bf3bac52b4, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:24.019533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dce8ab-2ced-4169-b201-3857b105a30c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:24.033985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1125/2247] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:23 2022] PUT /identity/v3/projects/f99b08d1a91f435c886dc463a9cfa85d/users/5e15b5b7f803454498a863bf3bac52b4/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:24.038168 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef03ff-9f64-4ed7-8f16-1d4e4dcd6e32 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:24.049121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef03ff-9f64-4ed7-8f16-1d4e4dcd6e32 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:24.115027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5b367-a454-4772-9b94-b09d1a8ad81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:24.115704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5b367-a454-4772-9b94-b09d1a8ad81d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:24.115993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5b367-a454-4772-9b94-b09d1a8ad81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:24.116270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5b367-a454-4772-9b94-b09d1a8ad81d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:24.117008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5b367-a454-4772-9b94-b09d1a8ad81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:24.118720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5b367-a454-4772-9b94-b09d1a8ad81d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:24.130246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1123/2248] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:24.134815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce8ab-2ced-4169-b201-3857b105a30c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:24.146349 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce8ab-2ced-4169-b201-3857b105a30c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:24.151786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a822afe-8a5f-4d79-b373-c1bf9b22c38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:24.152694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822afe-8a5f-4d79-b373-c1bf9b22c38b admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:24.152974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822afe-8a5f-4d79-b373-c1bf9b22c38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:24.153289 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a822afe-8a5f-4d79-b373-c1bf9b22c38b admin admin] PATH_INFO: `/v3/projects/f99b08d1a91f435c886dc463a9cfa85d/users/5e15b5b7f803454498a863bf3bac52b4/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:24.155473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a822afe-8a5f-4d79-b373-c1bf9b22c38b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f99b08d1a91f435c886dc463a9cfa85d, user_id=5e15b5b7f803454498a863bf3bac52b4, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:24.158287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a822afe-8a5f-4d79-b373-c1bf9b22c38b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:24.169904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1126/2249] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:24 2022] PUT /identity/v3/projects/f99b08d1a91f435c886dc463a9cfa85d/users/5e15b5b7f803454498a863bf3bac52b4/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:24.871000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444315-23e5-4452-b9a0-9b410ea9f254 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:24.871495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444315-23e5-4452-b9a0-9b410ea9f254 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:24.871601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9444315-23e5-4452-b9a0-9b410ea9f254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:24.906558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a9444315-23e5-4452-b9a0-9b410ea9f254 None None] MFA Rules not processed for user `5e15b5b7f803454498a863bf3bac52b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:24.908526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a9444315-23e5-4452-b9a0-9b410ea9f254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:24.987722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1124/2250] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:26.117971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45034701-b51e-415f-a2d6-46659d8362f7 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.118556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45034701-b51e-415f-a2d6-46659d8362f7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.118556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45034701-b51e-415f-a2d6-46659d8362f7 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.119940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1127/2251] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:24:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:24:26.129674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f106402-8ed7-45ad-820a-68bd2283cad4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.130197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f106402-8ed7-45ad-820a-68bd2283cad4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.130197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f106402-8ed7-45ad-820a-68bd2283cad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.150709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0f106402-8ed7-45ad-820a-68bd2283cad4 None None] MFA Rules not processed for user `7a2b40b05a6846099bd4c66609365ad5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:26.152086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f106402-8ed7-45ad-820a-68bd2283cad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:26.184612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45034701-b51e-415f-a2d6-46659d8362f7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:26.193077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:26.222755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:26.234106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1125/2252] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:24:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:26.245595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f106402-8ed7-45ad-820a-68bd2283cad4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:26.247010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f106402-8ed7-45ad-820a-68bd2283cad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:26.260550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f106402-8ed7-45ad-820a-68bd2283cad4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:26.262639 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45034701-b51e-415f-a2d6-46659d8362f7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:26.265955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73235b7-e9b6-4dc6-97ac-17e94df9245e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:26.266617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73235b7-e9b6-4dc6-97ac-17e94df9245e service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.266902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73235b7-e9b6-4dc6-97ac-17e94df9245e service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.267126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:26.267238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73235b7-e9b6-4dc6-97ac-17e94df9245e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.268001 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.268288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.268565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] PATH_INFO: `/v3/users/9b4bc4411ea9473080f88cf429869511` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.269787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b4bc4411ea9473080f88cf429869511)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:26.271718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:26.276498 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73235b7-e9b6-4dc6-97ac-17e94df9245e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:26.278269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73235b7-e9b6-4dc6-97ac-17e94df9245e service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:26.289237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1126/2253] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:24:26.324105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b4bc4411ea9473080f88cf429869511'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.336826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b4bc4411ea9473080f88cf429869511'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.337571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b4bc4411ea9473080f88cf429869511'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.345048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b4bc4411ea9473080f88cf429869511'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.352825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b4bc4411ea9473080f88cf429869511'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.359487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1128/2254] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:26 2022] DELETE /identity/v3/users/9b4bc4411ea9473080f88cf429869511 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:26.581250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73235b7-e9b6-4dc6-97ac-17e94df9245e service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:26.597938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73235b7-e9b6-4dc6-97ac-17e94df9245e service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:26.693482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:26.695510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.696306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.696532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] PATH_INFO: `/v3/projects/3058799973fc4d19a77910e259126579` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.698705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3058799973fc4d19a77910e259126579)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:26.702802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:26.804184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3058799973fc4d19a77910e259126579', 'request_id': 'req-4f4c1039-ee9a-477c-9051-544a129031d4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.814142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3058799973fc4d19a77910e259126579', 'request_id': 'req-4f4c1039-ee9a-477c-9051-544a129031d4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.826587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3058799973fc4d19a77910e259126579', 'request_id': 'req-4f4c1039-ee9a-477c-9051-544a129031d4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.851013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] The token cache is being invalidate because project 3058799973fc4d19a77910e259126579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:26.851545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.852901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:26.855701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1127/2255] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:26 2022] DELETE /identity/v3/projects/3058799973fc4d19a77910e259126579 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:26.861467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:26.863166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:26.878239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3058f279-baee-4f8c-b195-48cd852a8eab admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:26.960241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:26.961296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:26.961571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:26.961908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] PATH_INFO: `/v3/users/5e15b5b7f803454498a863bf3bac52b4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:26.963162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e15b5b7f803454498a863bf3bac52b4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:26.965373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.040016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e15b5b7f803454498a863bf3bac52b4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.043805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:27.060694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e15b5b7f803454498a863bf3bac52b4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.061587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e15b5b7f803454498a863bf3bac52b4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.071681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e15b5b7f803454498a863bf3bac52b4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.083832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8a6c6f55-00ae-4d19-a00f-8d39a8bf93a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e15b5b7f803454498a863bf3bac52b4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.093721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1129/2256] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:26 2022] DELETE /identity/v3/users/5e15b5b7f803454498a863bf3bac52b4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:24:27.327507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.330342 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:27.354712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c1039-ee9a-477c-9051-544a129031d4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.435324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:27.452848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.454335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.454880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.455472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] PATH_INFO: `/v3/projects/f99b08d1a91f435c886dc463a9cfa85d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.458330 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f99b08d1a91f435c886dc463a9cfa85d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.460490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.532116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f99b08d1a91f435c886dc463a9cfa85d', 'request_id': 'req-649a9e6d-0361-4436-863f-fc62a9c31dd6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.538055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f99b08d1a91f435c886dc463a9cfa85d', 'request_id': 'req-649a9e6d-0361-4436-863f-fc62a9c31dd6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.548585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f99b08d1a91f435c886dc463a9cfa85d', 'request_id': 'req-649a9e6d-0361-4436-863f-fc62a9c31dd6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.564156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] The token cache is being invalidate because project f99b08d1a91f435c886dc463a9cfa85d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:27.564467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.565104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-649a9e6d-0361-4436-863f-fc62a9c31dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:27.566732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1128/2257] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:27 2022] DELETE /identity/v3/projects/f99b08d1a91f435c886dc463a9cfa85d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:27.573478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0593-be7a-4f6e-8558-bd3c648379ac None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.574016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0593-be7a-4f6e-8558-bd3c648379ac None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.574016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af0593-be7a-4f6e-8558-bd3c648379ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.595176 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-80af0593-be7a-4f6e-8558-bd3c648379ac None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:27.596419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-80af0593-be7a-4f6e-8558-bd3c648379ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:27.674663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1130/2258] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:27.684030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b9b3a-581f-42c4-9cde-1624eed12a08 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.684312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b9b3a-581f-42c4-9cde-1624eed12a08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.684593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b9b3a-581f-42c4-9cde-1624eed12a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.707788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9b8b9b3a-581f-42c4-9cde-1624eed12a08 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:27.709017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9b8b9b3a-581f-42c4-9cde-1624eed12a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:27.717489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1129/2259] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:27.722636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af0593-be7a-4f6e-8558-bd3c648379ac None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.737028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80af0593-be7a-4f6e-8558-bd3c648379ac None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.744054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.744744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.745075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.745362 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.746123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:27.746434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.748059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.763125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1131/2260] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:27.767661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b9b3a-581f-42c4-9cde-1624eed12a08 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.780295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8b9b3a-581f-42c4-9cde-1624eed12a08 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.785856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90fc81-41f7-4fa6-9630-1a4e0a623d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.786613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90fc81-41f7-4fa6-9630-1a4e0a623d63 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.786892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90fc81-41f7-4fa6-9630-1a4e0a623d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.787169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90fc81-41f7-4fa6-9630-1a4e0a623d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.787912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90fc81-41f7-4fa6-9630-1a4e0a623d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.790336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90fc81-41f7-4fa6-9630-1a4e0a623d63 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.826778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1130/2261] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:27 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:27.831386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.843804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838b28b-c1a5-49f0-9ee3-432fd8647218 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.915306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e9216-3201-46d3-9eac-19480b6909da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.916124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e9216-3201-46d3-9eac-19480b6909da admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.916395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e9216-3201-46d3-9eac-19480b6909da admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.916684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e9216-3201-46d3-9eac-19480b6909da admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.917457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e9216-3201-46d3-9eac-19480b6909da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.919379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e9216-3201-46d3-9eac-19480b6909da admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:27.943847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1132/2262] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:27 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:27.948849 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90fc81-41f7-4fa6-9630-1a4e0a623d63 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:27.969834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90fc81-41f7-4fa6-9630-1a4e0a623d63 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:27.977752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bd0f4-4221-443a-93f0-d6cc09cfcd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:27.978497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bd0f4-4221-443a-93f0-d6cc09cfcd3f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:27.978841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bd0f4-4221-443a-93f0-d6cc09cfcd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:27.979195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760bd0f4-4221-443a-93f0-d6cc09cfcd3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:27.980486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bd0f4-4221-443a-93f0-d6cc09cfcd3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:27.982758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760bd0f4-4221-443a-93f0-d6cc09cfcd3f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.001028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1131/2263] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:28.002573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e9216-3201-46d3-9eac-19480b6909da admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:28.014997 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e9216-3201-46d3-9eac-19480b6909da admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:28.024116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e4a20-9549-4447-989c-a1a87f940220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:28.024992 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4a20-9549-4447-989c-a1a87f940220 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.025450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4a20-9549-4447-989c-a1a87f940220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.025968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e4a20-9549-4447-989c-a1a87f940220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.027542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4a20-9549-4447-989c-a1a87f940220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:28.029215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e4a20-9549-4447-989c-a1a87f940220 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.048326 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1133/2264] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:24:28.049972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bd0f4-4221-443a-93f0-d6cc09cfcd3f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:28.063417 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760bd0f4-4221-443a-93f0-d6cc09cfcd3f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:28.070485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:28.071279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.071591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.071882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] PATH_INFO: `/v3/projects/a9ce3f00870c44c4a6ea7d8c3cab97a7/users/4e67af51e1f5479a845cd8a0f2f4235c/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.096880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9ce3f00870c44c4a6ea7d8c3cab97a7, user_id=4e67af51e1f5479a845cd8a0f2f4235c, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:28.100403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.123914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1132/2265] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:28 2022] PUT /identity/v3/projects/a9ce3f00870c44c4a6ea7d8c3cab97a7/users/4e67af51e1f5479a845cd8a0f2f4235c/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:28.134765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a91114-4f5e-4f7d-956f-f913cf20ea86 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.135153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a91114-4f5e-4f7d-956f-f913cf20ea86 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.135510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a91114-4f5e-4f7d-956f-f913cf20ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.170690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-32a91114-4f5e-4f7d-956f-f913cf20ea86 None None] MFA Rules not processed for user `4e67af51e1f5479a845cd8a0f2f4235c`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:28.172109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-32a91114-4f5e-4f7d-956f-f913cf20ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:28.196964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:28.255583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1134/2266] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:28.268026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:28.270680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:28.340160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dced3b-8917-442a-86a5-b7eb8d8b8fdf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:28.344926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f486863-bd8c-4652-821e-805765ba71a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:28.345655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f486863-bd8c-4652-821e-805765ba71a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.345924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f486863-bd8c-4652-821e-805765ba71a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.346208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f486863-bd8c-4652-821e-805765ba71a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.356166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f486863-bd8c-4652-821e-805765ba71a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:28.357969 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f486863-bd8c-4652-821e-805765ba71a7 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:28.368141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1133/2267] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:28.859896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98050d-f215-4122-acbe-85554bb77aa8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:28.860385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98050d-f215-4122-acbe-85554bb77aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:28.860476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98050d-f215-4122-acbe-85554bb77aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:28.879204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-fa98050d-f215-4122-acbe-85554bb77aa8 None None] MFA Rules not processed for user `da4b12d5d8f647589a2226303b01c2f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:28.880330 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-fa98050d-f215-4122-acbe-85554bb77aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:28.949570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1135/2268] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:38.363107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:38.942186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=42 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:42.436714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f486863-bd8c-4652-821e-805765ba71a7 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:42.521419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f486863-bd8c-4652-821e-805765ba71a7 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:42.527127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:42.528018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:42.528332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:42.528603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] PATH_INFO: `/v3/users/4e67af51e1f5479a845cd8a0f2f4235c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:42.529845 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e67af51e1f5479a845cd8a0f2f4235c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:42.531965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:42.597654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e67af51e1f5479a845cd8a0f2f4235c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:42.598400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e67af51e1f5479a845cd8a0f2f4235c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:42.599043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e67af51e1f5479a845cd8a0f2f4235c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:42.615847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e67af51e1f5479a845cd8a0f2f4235c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:42.627682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e67af51e1f5479a845cd8a0f2f4235c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:42.644513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1134/2269] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:42 2022] DELETE /identity/v3/users/4e67af51e1f5479a845cd8a0f2f4235c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:24:42.650401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99b79-c5a7-472b-9b05-3794cfa6dbae None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:42.651526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99b79-c5a7-472b-9b05-3794cfa6dbae None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:42.651526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f99b79-c5a7-472b-9b05-3794cfa6dbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:42.674279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-39f99b79-c5a7-472b-9b05-3794cfa6dbae None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:42.675572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-39f99b79-c5a7-472b-9b05-3794cfa6dbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:42.760973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1136/2270] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:42.776447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:42.778561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:42.910421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7dc2e-5cbf-49dd-8a26-df399528ccf5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:42.918054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a62f-0a0f-44dd-92d1-ebc126730921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:42.919572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a62f-0a0f-44dd-92d1-ebc126730921 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:42.920427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a62f-0a0f-44dd-92d1-ebc126730921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:42.922534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a62f-0a0f-44dd-92d1-ebc126730921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:42.942523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78a62f-0a0f-44dd-92d1-ebc126730921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:42.945292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78a62f-0a0f-44dd-92d1-ebc126730921 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:42.960610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1135/2271] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:43.279620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f99b79-c5a7-472b-9b05-3794cfa6dbae None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.281116 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-39f99b79-c5a7-472b-9b05-3794cfa6dbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:43.299603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f99b79-c5a7-472b-9b05-3794cfa6dbae None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.307273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.308712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.309416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.309964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] PATH_INFO: `/v3/projects/a9ce3f00870c44c4a6ea7d8c3cab97a7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.312134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9ce3f00870c44c4a6ea7d8c3cab97a7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:43.315833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:43.375436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9ce3f00870c44c4a6ea7d8c3cab97a7', 'request_id': 'req-78f14603-3d90-4d6a-94a7-0f009e559d13'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.387991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9ce3f00870c44c4a6ea7d8c3cab97a7', 'request_id': 'req-78f14603-3d90-4d6a-94a7-0f009e559d13'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.396656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9ce3f00870c44c4a6ea7d8c3cab97a7', 'request_id': 'req-78f14603-3d90-4d6a-94a7-0f009e559d13'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.410568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] The token cache is being invalidate because project a9ce3f00870c44c4a6ea7d8c3cab97a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:43.410860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.411657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.413666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1137/2272] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:43 2022] DELETE /identity/v3/projects/a9ce3f00870c44c4a6ea7d8c3cab97a7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:43.419381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950fec0-25af-4bb7-b4ac-33accd904dc9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.419673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950fec0-25af-4bb7-b4ac-33accd904dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.419954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950fec0-25af-4bb7-b4ac-33accd904dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.440062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b950fec0-25af-4bb7-b4ac-33accd904dc9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:43.441373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b950fec0-25af-4bb7-b4ac-33accd904dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:43.471776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:43.524849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.526469 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:43.538961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1136/2273] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:43.548719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83777a5-e226-463b-bbc4-56916df04453 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.549094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83777a5-e226-463b-bbc4-56916df04453 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.550835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83777a5-e226-463b-bbc4-56916df04453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.577756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e83777a5-e226-463b-bbc4-56916df04453 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:43.579262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e83777a5-e226-463b-bbc4-56916df04453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:43.588993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1137/2274] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:43.593491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83777a5-e226-463b-bbc4-56916df04453 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.604003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83777a5-e226-463b-bbc4-56916df04453 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.609566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.610211 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.610485 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.610589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f14603-3d90-4d6a-94a7-0f009e559d13 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.610762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.611495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:43.611804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:43.613444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:43.615860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.616795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.617150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.617645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] PATH_INFO: `/v3/users/a449ba1022f345fea6484e5ae173b3e2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.619187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a449ba1022f345fea6484e5ae173b3e2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:43.621104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:43.629902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1138/2275] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:24:43.634022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.644570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666895b-b596-480e-996e-1d8ca1752e47 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.651086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.651903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.652210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.652568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.653367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:43.655582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:43.656488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:43.688270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a449ba1022f345fea6484e5ae173b3e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.699152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1139/2276] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:43 2022] POST /identity/v3/projects => generated 375 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:43.703347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a449ba1022f345fea6484e5ae173b3e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.703548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.704131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a449ba1022f345fea6484e5ae173b3e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.705515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:43.713069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a449ba1022f345fea6484e5ae173b3e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.723873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c2e3987b-b88b-4a0c-b9b2-416c24cc5d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a449ba1022f345fea6484e5ae173b3e2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:43.731859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1138/2277] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:43 2022] DELETE /identity/v3/users/a449ba1022f345fea6484e5ae173b3e2 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:24:43.737588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9124b-ccc2-44db-ad2f-5e50d05e6436 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.737966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9124b-ccc2-44db-ad2f-5e50d05e6436 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.738353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9124b-ccc2-44db-ad2f-5e50d05e6436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.760798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-06f9124b-ccc2-44db-ad2f-5e50d05e6436 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:43.762406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-06f9124b-ccc2-44db-ad2f-5e50d05e6436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:43.782670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bd9a3-2513-4efe-b32c-2b39ace1c860 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.849562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1139/2278] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:43.853281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfd1f6-8cb5-43bd-a593-8985f58026e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.854215 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfd1f6-8cb5-43bd-a593-8985f58026e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.854478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfd1f6-8cb5-43bd-a593-8985f58026e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.854683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adfd1f6-8cb5-43bd-a593-8985f58026e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.855353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfd1f6-8cb5-43bd-a593-8985f58026e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:43.857346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adfd1f6-8cb5-43bd-a593-8985f58026e6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:43.861139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9124b-ccc2-44db-ad2f-5e50d05e6436 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.863377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-06f9124b-ccc2-44db-ad2f-5e50d05e6436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:43.881231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1140/2279] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:43 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:43.885688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfd1f6-8cb5-43bd-a593-8985f58026e6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.900750 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adfd1f6-8cb5-43bd-a593-8985f58026e6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.912297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cc803-a85b-46a2-9190-c7cd1fc5f7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.912297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cc803-a85b-46a2-9190-c7cd1fc5f7ae admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.912297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cc803-a85b-46a2-9190-c7cd1fc5f7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.912297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cc803-a85b-46a2-9190-c7cd1fc5f7ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.912759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cc803-a85b-46a2-9190-c7cd1fc5f7ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:43.912759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cc803-a85b-46a2-9190-c7cd1fc5f7ae admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:43.923403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1141/2280] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:24:43.927583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cc803-a85b-46a2-9190-c7cd1fc5f7ae admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.938992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cc803-a85b-46a2-9190-c7cd1fc5f7ae admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.943871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dba1-b755-41af-ad55-409fece42010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.944563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dba1-b755-41af-ad55-409fece42010 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.944847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dba1-b755-41af-ad55-409fece42010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.945231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dba1-b755-41af-ad55-409fece42010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.945610 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9124b-ccc2-44db-ad2f-5e50d05e6436 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.946372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250dba1-b755-41af-ad55-409fece42010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:43.948070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250dba1-b755-41af-ad55-409fece42010 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:43.949876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:43.950393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87441855-3cc9-4c0c-a599-c7fde6af51b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.951098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87441855-3cc9-4c0c-a599-c7fde6af51b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.951386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87441855-3cc9-4c0c-a599-c7fde6af51b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.951666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87441855-3cc9-4c0c-a599-c7fde6af51b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:43.964423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87441855-3cc9-4c0c-a599-c7fde6af51b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:43.966364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1142/2281] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:24:43.967226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87441855-3cc9-4c0c-a599-c7fde6af51b3 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:43.970337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dba1-b755-41af-ad55-409fece42010 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:43.985209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1140/2282] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:24:43.991453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dba1-b755-41af-ad55-409fece42010 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:43.996955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467327a-ed66-4100-af0f-4d626762644c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:43.998050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467327a-ed66-4100-af0f-4d626762644c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:43.998669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467327a-ed66-4100-af0f-4d626762644c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:43.999043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467327a-ed66-4100-af0f-4d626762644c admin admin] PATH_INFO: `/v3/projects/f0299e30cf9941278ebc7fb4316828b5/users/b63c2ad3d66e42c79ce993ee828fce08/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:44.035173 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467327a-ed66-4100-af0f-4d626762644c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0299e30cf9941278ebc7fb4316828b5, user_id=b63c2ad3d66e42c79ce993ee828fce08, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:44.038943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467327a-ed66-4100-af0f-4d626762644c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:44.073748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1143/2283] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:43 2022] PUT /identity/v3/projects/f0299e30cf9941278ebc7fb4316828b5/users/b63c2ad3d66e42c79ce993ee828fce08/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:44.087486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402fc61-2703-4b72-8248-99fec4e9ba05 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:44.089007 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402fc61-2703-4b72-8248-99fec4e9ba05 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:44.089557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402fc61-2703-4b72-8248-99fec4e9ba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:44.125772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d402fc61-2703-4b72-8248-99fec4e9ba05 None None] MFA Rules not processed for user `b63c2ad3d66e42c79ce993ee828fce08`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:44.127131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d402fc61-2703-4b72-8248-99fec4e9ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:44.182552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:44.284880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1141/2284] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:44.291922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467327a-ed66-4100-af0f-4d626762644c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:44.472945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467327a-ed66-4100-af0f-4d626762644c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:44.484753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a01965-a89a-4f85-a505-36cfd843035c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:44.485377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a01965-a89a-4f85-a505-36cfd843035c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:44.487082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a01965-a89a-4f85-a505-36cfd843035c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:44.487915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a01965-a89a-4f85-a505-36cfd843035c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:44.489051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a01965-a89a-4f85-a505-36cfd843035c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:44.492802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a01965-a89a-4f85-a505-36cfd843035c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:44.499798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402fc61-2703-4b72-8248-99fec4e9ba05 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:44.501643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d402fc61-2703-4b72-8248-99fec4e9ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:44.550622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402fc61-2703-4b72-8248-99fec4e9ba05 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:44.568629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:44.571067 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:44.571895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:44.572611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] PATH_INFO: `/v3/projects/2725917e195e47fbb93e5d9e7b1e8802` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:44.575586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2725917e195e47fbb93e5d9e7b1e8802)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:44.588821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:44.592752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1144/2285] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:44 2022] POST /identity/v3/projects => generated 375 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:44.596297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a01965-a89a-4f85-a505-36cfd843035c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:44.746322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2725917e195e47fbb93e5d9e7b1e8802', 'request_id': 'req-dd8dc67b-f414-4443-8627-a4c1e8c0b049'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:44.756948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2725917e195e47fbb93e5d9e7b1e8802', 'request_id': 'req-dd8dc67b-f414-4443-8627-a4c1e8c0b049'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:44.767548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2725917e195e47fbb93e5d9e7b1e8802', 'request_id': 'req-dd8dc67b-f414-4443-8627-a4c1e8c0b049'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:44.773101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a01965-a89a-4f85-a505-36cfd843035c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:44.784824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] The token cache is being invalidate because project 2725917e195e47fbb93e5d9e7b1e8802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:44.785292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:44.786916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:44.789988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1142/2286] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:44 2022] DELETE /identity/v3/projects/2725917e195e47fbb93e5d9e7b1e8802 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:44.897213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507a8a1-7925-4047-8c32-0619d5bc82b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:44.897213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507a8a1-7925-4047-8c32-0619d5bc82b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:44.897213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507a8a1-7925-4047-8c32-0619d5bc82b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:44.897213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507a8a1-7925-4047-8c32-0619d5bc82b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:44.902069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507a8a1-7925-4047-8c32-0619d5bc82b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:44.904329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0507a8a1-7925-4047-8c32-0619d5bc82b5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:44.918563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:44.935938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1145/2287] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:44 2022] POST /identity/v3/users => generated 294 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:44.940025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:44.941693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:44.963865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8dc67b-f414-4443-8627-a4c1e8c0b049 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:44.972002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84833f53-94e8-40c3-9e6d-57fc2bfb2f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:44.972716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84833f53-94e8-40c3-9e6d-57fc2bfb2f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:44.973058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84833f53-94e8-40c3-9e6d-57fc2bfb2f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:44.973303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84833f53-94e8-40c3-9e6d-57fc2bfb2f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:44.974058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84833f53-94e8-40c3-9e6d-57fc2bfb2f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:44.977222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84833f53-94e8-40c3-9e6d-57fc2bfb2f9f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:44.997548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1143/2288] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:44.998944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507a8a1-7925-4047-8c32-0619d5bc82b5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.012255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0507a8a1-7925-4047-8c32-0619d5bc82b5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.019133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c26be-23b7-4c3f-8cf7-233b9bfec7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.020306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c26be-23b7-4c3f-8cf7-233b9bfec7a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.020680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c26be-23b7-4c3f-8cf7-233b9bfec7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.021046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c26be-23b7-4c3f-8cf7-233b9bfec7a9 admin admin] PATH_INFO: `/v3/projects/8579e046969e405bb8513ceb9298cfe6/users/78c58d2b56c64171b73963a968449fea/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.038617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c26be-23b7-4c3f-8cf7-233b9bfec7a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8579e046969e405bb8513ceb9298cfe6, user_id=78c58d2b56c64171b73963a968449fea, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:45.043471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c26be-23b7-4c3f-8cf7-233b9bfec7a9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:45.062113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1146/2289] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:44 2022] PUT /identity/v3/projects/8579e046969e405bb8513ceb9298cfe6/users/78c58d2b56c64171b73963a968449fea/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:24:45.066028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84833f53-94e8-40c3-9e6d-57fc2bfb2f9f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.085657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84833f53-94e8-40c3-9e6d-57fc2bfb2f9f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.180513 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19e9b0-6d89-4501-b6c9-0fb95cc7d350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.181516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e9b0-6d89-4501-b6c9-0fb95cc7d350 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.181939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e9b0-6d89-4501-b6c9-0fb95cc7d350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.182226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19e9b0-6d89-4501-b6c9-0fb95cc7d350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.182998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19e9b0-6d89-4501-b6c9-0fb95cc7d350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:45.184651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19e9b0-6d89-4501-b6c9-0fb95cc7d350 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:45.200629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1144/2290] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:45.205317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c26be-23b7-4c3f-8cf7-233b9bfec7a9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.227350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c26be-23b7-4c3f-8cf7-233b9bfec7a9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.237483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.238646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.241831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.242841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] PATH_INFO: `/v3/projects/8579e046969e405bb8513ceb9298cfe6/users/78c58d2b56c64171b73963a968449fea/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.247067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8579e046969e405bb8513ceb9298cfe6, user_id=78c58d2b56c64171b73963a968449fea, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:45.250112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:45.261931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1147/2291] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:45 2022] PUT /identity/v3/projects/8579e046969e405bb8513ceb9298cfe6/users/78c58d2b56c64171b73963a968449fea/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:45.272894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ac2c-e04d-4058-95fd-5bb4dacde271 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.273240 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ac2c-e04d-4058-95fd-5bb4dacde271 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.273552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075ac2c-e04d-4058-95fd-5bb4dacde271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.309918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7075ac2c-e04d-4058-95fd-5bb4dacde271 None None] MFA Rules not processed for user `78c58d2b56c64171b73963a968449fea`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:45.311082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7075ac2c-e04d-4058-95fd-5bb4dacde271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:45.354681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:45.403827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1145/2292] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:45.416822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:45.418916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:45.499672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01ece2a-b15e-4507-ab70-5cbad4716cf9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:45.507066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546cb91e-f57d-4a87-a113-074ff7cf3436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:45.507803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cb91e-f57d-4a87-a113-074ff7cf3436 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:45.508610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cb91e-f57d-4a87-a113-074ff7cf3436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:45.508610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546cb91e-f57d-4a87-a113-074ff7cf3436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:45.510888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-546cb91e-f57d-4a87-a113-074ff7cf3436 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:45.579353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cb91e-f57d-4a87-a113-074ff7cf3436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:45.581000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546cb91e-f57d-4a87-a113-074ff7cf3436 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:45.591541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1148/2293] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:48.218479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ac2c-e04d-4058-95fd-5bb4dacde271 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:48.312559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075ac2c-e04d-4058-95fd-5bb4dacde271 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:48.318138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:48.319125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:48.319471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:48.319812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] PATH_INFO: `/v3/users/b63c2ad3d66e42c79ce993ee828fce08` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:48.322216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63c2ad3d66e42c79ce993ee828fce08)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:48.324176 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:48.394577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63c2ad3d66e42c79ce993ee828fce08'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:48.415692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63c2ad3d66e42c79ce993ee828fce08'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:48.417237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63c2ad3d66e42c79ce993ee828fce08'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:48.451359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63c2ad3d66e42c79ce993ee828fce08'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:48.467591 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63c2ad3d66e42c79ce993ee828fce08'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:48.490904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1146/2294] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:48 2022] DELETE /identity/v3/users/b63c2ad3d66e42c79ce993ee828fce08 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:48.491133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0b617-0b71-4b47-a878-29d866f57a4f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:48.491133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0b617-0b71-4b47-a878-29d866f57a4f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:48.491133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0b617-0b71-4b47-a878-29d866f57a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:48.560497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bdf0b617-0b71-4b47-a878-29d866f57a4f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:48.561792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bdf0b617-0b71-4b47-a878-29d866f57a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:48.612101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:48.700630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1149/2295] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:48.713773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:48.716884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:48.811678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:48.891727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58c10-18e9-4d7f-98e9-da495a0b77d6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:48.907343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed74b7-e5f4-4fa4-9906-617cddb9c372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:48.908169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed74b7-e5f4-4fa4-9906-617cddb9c372 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:48.908521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed74b7-e5f4-4fa4-9906-617cddb9c372 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:48.908866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed74b7-e5f4-4fa4-9906-617cddb9c372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:48.920655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed74b7-e5f4-4fa4-9906-617cddb9c372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:48.922422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed74b7-e5f4-4fa4-9906-617cddb9c372 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:48.937263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1147/2296] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:49.347355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0b617-0b71-4b47-a878-29d866f57a4f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:49.348967 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bdf0b617-0b71-4b47-a878-29d866f57a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:49.362605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0b617-0b71-4b47-a878-29d866f57a4f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:49.368164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:49.369238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:49.369513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:49.369787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] PATH_INFO: `/v3/projects/f0299e30cf9941278ebc7fb4316828b5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:49.371183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0299e30cf9941278ebc7fb4316828b5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:49.373141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:49.444465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0299e30cf9941278ebc7fb4316828b5', 'request_id': 'req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.451103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0299e30cf9941278ebc7fb4316828b5', 'request_id': 'req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.459926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0299e30cf9941278ebc7fb4316828b5', 'request_id': 'req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.474056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] The token cache is being invalidate because project f0299e30cf9941278ebc7fb4316828b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:49.474319 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.475443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.477718 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1150/2297] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:49 2022] DELETE /identity/v3/projects/f0299e30cf9941278ebc7fb4316828b5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:49.482007 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed74b7-e5f4-4fa4-9906-617cddb9c372 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:49.492870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed74b7-e5f4-4fa4-9906-617cddb9c372 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:49.580908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:49.581837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:49.582121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:49.582420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] PATH_INFO: `/v3/users/78c58d2b56c64171b73963a968449fea` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:49.583662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c58d2b56c64171b73963a968449fea)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:49.585611 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:49.641843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c58d2b56c64171b73963a968449fea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.657817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c58d2b56c64171b73963a968449fea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.658576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c58d2b56c64171b73963a968449fea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.666612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c58d2b56c64171b73963a968449fea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.675569 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-e08236e7-90bb-4c7c-9fdb-057c00899973 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c58d2b56c64171b73963a968449fea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:49.682293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1148/2298] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:49 2022] DELETE /identity/v3/users/78c58d2b56c64171b73963a968449fea => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:49.996186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:49.998333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:50.014913 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa06a9-1c25-4e17-80ef-4b3ea6fc5d10 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:50.079343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:50.080231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.080510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.080785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] PATH_INFO: `/v3/projects/8579e046969e405bb8513ceb9298cfe6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.082085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8579e046969e405bb8513ceb9298cfe6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:50.084057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:50.123459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:50.148166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8579e046969e405bb8513ceb9298cfe6', 'request_id': 'req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:50.155119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8579e046969e405bb8513ceb9298cfe6', 'request_id': 'req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:50.163445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8579e046969e405bb8513ceb9298cfe6', 'request_id': 'req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:50.178335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] The token cache is being invalidate because project 8579e046969e405bb8513ceb9298cfe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:50.178442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:50.181681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8ea7b95f-e903-46ee-a0cd-55cbce6e19f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:50.183437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1151/2299] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:49 2022] DELETE /identity/v3/projects/8579e046969e405bb8513ceb9298cfe6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:50.188665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904f23a-a40b-4d6e-b06a-5119edd79e32 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.188952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904f23a-a40b-4d6e-b06a-5119edd79e32 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.189259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904f23a-a40b-4d6e-b06a-5119edd79e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.234699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b904f23a-a40b-4d6e-b06a-5119edd79e32 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:50.236100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b904f23a-a40b-4d6e-b06a-5119edd79e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:50.252655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:50.315219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1149/2300] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:50.325356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054527b6-e031-4317-a880-3aace8c81171 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.325620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054527b6-e031-4317-a880-3aace8c81171 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.325837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054527b6-e031-4317-a880-3aace8c81171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.346314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-054527b6-e031-4317-a880-3aace8c81171 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:50.347549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-054527b6-e031-4317-a880-3aace8c81171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:50.368863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1152/2301] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:50.374026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904f23a-a40b-4d6e-b06a-5119edd79e32 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:50.387451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904f23a-a40b-4d6e-b06a-5119edd79e32 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:50.400182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:50.401366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.401821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.402102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.403159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:50.403866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:50.406368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:50.423840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1150/2302] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:50.428504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054527b6-e031-4317-a880-3aace8c81171 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:50.439556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054527b6-e031-4317-a880-3aace8c81171 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:50.444990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89883-9fb2-4f28-a7d7-569ecbe43f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:50.445738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89883-9fb2-4f28-a7d7-569ecbe43f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.446022 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89883-9fb2-4f28-a7d7-569ecbe43f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.446299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b89883-9fb2-4f28-a7d7-569ecbe43f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.447027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b89883-9fb2-4f28-a7d7-569ecbe43f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:50.448977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b89883-9fb2-4f28-a7d7-569ecbe43f69 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:50.510711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1153/2303] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:50 2022] POST /identity/v3/projects => generated 381 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:24:50.516088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:50.528938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05b0413-6868-4e6e-a2e2-d1de50b61355 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:50.633364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45758a05-4305-40af-b76c-6ec91c32a448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:50.634231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45758a05-4305-40af-b76c-6ec91c32a448 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.635983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45758a05-4305-40af-b76c-6ec91c32a448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.636524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45758a05-4305-40af-b76c-6ec91c32a448 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.637850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45758a05-4305-40af-b76c-6ec91c32a448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:50.640885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45758a05-4305-40af-b76c-6ec91c32a448 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:50.672090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1151/2304] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:50 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:50.676744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89883-9fb2-4f28-a7d7-569ecbe43f69 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:50.689799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b89883-9fb2-4f28-a7d7-569ecbe43f69 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:50.696383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3e71e-b94f-4236-a40e-c60800ce180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:50.697103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e71e-b94f-4236-a40e-c60800ce180a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.698052 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e71e-b94f-4236-a40e-c60800ce180a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.698339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3e71e-b94f-4236-a40e-c60800ce180a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.699086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3e71e-b94f-4236-a40e-c60800ce180a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:50.700683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3e71e-b94f-4236-a40e-c60800ce180a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:50.715063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1154/2305] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:50.720119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45758a05-4305-40af-b76c-6ec91c32a448 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:50.734776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45758a05-4305-40af-b76c-6ec91c32a448 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:50.741374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f86c2-5d65-4771-ac34-0ef4bcc1a700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:50.742613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f86c2-5d65-4771-ac34-0ef4bcc1a700 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.743099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f86c2-5d65-4771-ac34-0ef4bcc1a700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.743384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f86c2-5d65-4771-ac34-0ef4bcc1a700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.745062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f86c2-5d65-4771-ac34-0ef4bcc1a700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:50.748086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f86c2-5d65-4771-ac34-0ef4bcc1a700 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:50.772019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1152/2306] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:50.778241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3e71e-b94f-4236-a40e-c60800ce180a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:50.790173 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3e71e-b94f-4236-a40e-c60800ce180a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:50.795587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:50.796653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.797016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.797433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] PATH_INFO: `/v3/projects/f96239e8e212410aac9df2d35293c665/users/8b088ae5ce5046ae83d9ae151dc52eca/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.817514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96239e8e212410aac9df2d35293c665, user_id=8b088ae5ce5046ae83d9ae151dc52eca, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:50.820246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:50.845415 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1155/2307] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:50 2022] PUT /identity/v3/projects/f96239e8e212410aac9df2d35293c665/users/8b088ae5ce5046ae83d9ae151dc52eca/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:50.851039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda91967-fbf7-4fc5-adb7-f0b51862bf61 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:50.851410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda91967-fbf7-4fc5-adb7-f0b51862bf61 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:50.852088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda91967-fbf7-4fc5-adb7-f0b51862bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:50.885879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cda91967-fbf7-4fc5-adb7-f0b51862bf61 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:50.885879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cda91967-fbf7-4fc5-adb7-f0b51862bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:50.970141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1153/2308] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:51.003779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:51.008086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:51.111902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df4efe-5b28-4e7f-95d9-6866a818929d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:51.116772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:51.117941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:51.118225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:51.118535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:51.154317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:51.154317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:51.165216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1156/2309] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:24:52.686529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083b75c-b4ce-4247-96e4-8236ba7b77c4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:52.687687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083b75c-b4ce-4247-96e4-8236ba7b77c4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:52.687995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083b75c-b4ce-4247-96e4-8236ba7b77c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:52.737477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7083b75c-b4ce-4247-96e4-8236ba7b77c4 None None] MFA Rules not processed for user `8b088ae5ce5046ae83d9ae151dc52eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:52.738945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7083b75c-b4ce-4247-96e4-8236ba7b77c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:52.754396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:52.892454 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1154/2310] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:52.921629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:52.924366 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:53.069065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dade6e9-bf04-4561-b182-baa3f9578b40 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:53.075358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:53.076478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:53.076863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:53.077284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:53.088580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:53.090056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:53.095530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:53.113376 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1157/2311] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:53.118192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b4ff-2a3f-4ad8-9a04-34f8e7932acd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:53.118489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b4ff-2a3f-4ad8-9a04-34f8e7932acd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:53.118773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0b4ff-2a3f-4ad8-9a04-34f8e7932acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:53.141514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3eb0b4ff-2a3f-4ad8-9a04-34f8e7932acd None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:53.143481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3eb0b4ff-2a3f-4ad8-9a04-34f8e7932acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:53.228981 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1155/2312] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:24:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:53.278433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:53.280078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:53.408088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee999c-f893-475d-a6f6-597b66e14a4e service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:53.423797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b40535b-abcf-4adc-a322-0b1fbedabc27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:53.426015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40535b-abcf-4adc-a322-0b1fbedabc27 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:53.426679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40535b-abcf-4adc-a322-0b1fbedabc27 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:53.428104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b40535b-abcf-4adc-a322-0b1fbedabc27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:53.447268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40535b-abcf-4adc-a322-0b1fbedabc27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:53.448616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b40535b-abcf-4adc-a322-0b1fbedabc27 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:53.486694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1158/2313] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:24:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:24:54.140195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc942d-c20d-43ca-b5eb-ba2f12706044 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.140704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc942d-c20d-43ca-b5eb-ba2f12706044 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.140928 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc942d-c20d-43ca-b5eb-ba2f12706044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.169953 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6bdc942d-c20d-43ca-b5eb-ba2f12706044 None None] MFA Rules not processed for user `0cd8a309f6914d83b8e72bc60a5d7924`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:54.172776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6bdc942d-c20d-43ca-b5eb-ba2f12706044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:54.292629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1156/2314] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:54.397171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8792b-1c9c-46ab-8aa3-ec9f73470509 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.397889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8792b-1c9c-46ab-8aa3-ec9f73470509 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.398529 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac8792b-1c9c-46ab-8aa3-ec9f73470509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.429714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2ac8792b-1c9c-46ab-8aa3-ec9f73470509 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:54.431495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2ac8792b-1c9c-46ab-8aa3-ec9f73470509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:54.441923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1159/2315] 10.222.0.66 () {58 vars in 967 bytes} [Thu Jun 16 17:24:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:54.460535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc942d-c20d-43ca-b5eb-ba2f12706044 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:54.483795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc942d-c20d-43ca-b5eb-ba2f12706044 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:54.491575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ce3a36-41f1-4a74-88b6-1a9af90795e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:54.492584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce3a36-41f1-4a74-88b6-1a9af90795e1 service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.493087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce3a36-41f1-4a74-88b6-1a9af90795e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.493584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ce3a36-41f1-4a74-88b6-1a9af90795e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.503798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce3a36-41f1-4a74-88b6-1a9af90795e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:54.505873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ce3a36-41f1-4a74-88b6-1a9af90795e1 service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:54.517680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1157/2316] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:24:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:24:54.746139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8ee4d-4724-4a14-957b-080063d8bb39 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.746753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8ee4d-4724-4a14-957b-080063d8bb39 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.747125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae8ee4d-4724-4a14-957b-080063d8bb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.779036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1ae8ee4d-4724-4a14-957b-080063d8bb39 None None] MFA Rules not processed for user `8b088ae5ce5046ae83d9ae151dc52eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:54.780390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ae8ee4d-4724-4a14-957b-080063d8bb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:54.799604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1160/2317] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:54.877119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1f53-334a-4c43-8ae9-eae2cf60fb3b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.878032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1f53-334a-4c43-8ae9-eae2cf60fb3b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.878568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b1f53-334a-4c43-8ae9-eae2cf60fb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.907652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cc0b1f53-334a-4c43-8ae9-eae2cf60fb3b None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:54.909129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cc0b1f53-334a-4c43-8ae9-eae2cf60fb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:54.925432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1158/2318] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:24:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:54.946068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8ee4d-4724-4a14-957b-080063d8bb39 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:54.959407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae8ee4d-4724-4a14-957b-080063d8bb39 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:54.965965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1b393-e29f-4d4e-884f-ea7fd042970c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:54.966650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1b393-e29f-4d4e-884f-ea7fd042970c service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:54.966952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1b393-e29f-4d4e-884f-ea7fd042970c service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:54.967392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c1b393-e29f-4d4e-884f-ea7fd042970c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:54.977646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1b393-e29f-4d4e-884f-ea7fd042970c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:54.978805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c1b393-e29f-4d4e-884f-ea7fd042970c service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:54.990838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1161/2319] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:24:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 17:24:55.526350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c7a8-dcbd-4946-a8af-d0551e2f29db None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:55.527555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c7a8-dcbd-4946-a8af-d0551e2f29db None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:55.528232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b98c7a8-dcbd-4946-a8af-d0551e2f29db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:55.567457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7b98c7a8-dcbd-4946-a8af-d0551e2f29db None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:55.568730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7b98c7a8-dcbd-4946-a8af-d0551e2f29db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:55.585254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1159/2320] 10.222.0.66 () {58 vars in 967 bytes} [Thu Jun 16 17:24:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:55.608958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1b393-e29f-4d4e-884f-ea7fd042970c service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:55.623271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c1b393-e29f-4d4e-884f-ea7fd042970c service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:55.629764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd3efac-467f-4343-b322-cb9eb421d8f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:55.630657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3efac-467f-4343-b322-cb9eb421d8f0 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:55.630949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3efac-467f-4343-b322-cb9eb421d8f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:55.631299 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd3efac-467f-4343-b322-cb9eb421d8f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:55.645871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3efac-467f-4343-b322-cb9eb421d8f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:55.649007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd3efac-467f-4343-b322-cb9eb421d8f0 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:55.664788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1162/2321] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:24:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:24:58.243492 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98c7a8-dcbd-4946-a8af-d0551e2f29db None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:58.245718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7b98c7a8-dcbd-4946-a8af-d0551e2f29db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:58.260898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b98c7a8-dcbd-4946-a8af-d0551e2f29db None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:58.266958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:58.268074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:58.268342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:58.268865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] PATH_INFO: `/v3/users/da4b12d5d8f647589a2226303b01c2f3` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:58.270224 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da4b12d5d8f647589a2226303b01c2f3)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:58.272243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:58.321433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:58.350524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da4b12d5d8f647589a2226303b01c2f3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:58.373530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da4b12d5d8f647589a2226303b01c2f3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:58.374542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da4b12d5d8f647589a2226303b01c2f3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:58.394133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da4b12d5d8f647589a2226303b01c2f3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:58.413889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da4b12d5d8f647589a2226303b01c2f3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:58.422075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1160/2322] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:24:58 2022] DELETE /identity/v3/users/da4b12d5d8f647589a2226303b01c2f3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:58.428605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c9cd-1588-41ba-88bd-2838af5f0fd7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:58.430129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c9cd-1588-41ba-88bd-2838af5f0fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:58.430778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0c9cd-1588-41ba-88bd-2838af5f0fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:58.452286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-abe0c9cd-1588-41ba-88bd-2838af5f0fd7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:58.453951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-abe0c9cd-1588-41ba-88bd-2838af5f0fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:58.555471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1163/2323] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:58.566577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:58.568924 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:58.643313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c50d8d-f380-49cf-969a-2f8115d37382 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:58.658941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d97662-507b-4774-98fd-d10c9220da38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:58.660796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97662-507b-4774-98fd-d10c9220da38 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:58.660796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97662-507b-4774-98fd-d10c9220da38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:58.662047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d97662-507b-4774-98fd-d10c9220da38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:58.678711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d97662-507b-4774-98fd-d10c9220da38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:58.680405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d97662-507b-4774-98fd-d10c9220da38 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:58.694724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1161/2324] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:24:58.903496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0c9cd-1588-41ba-88bd-2838af5f0fd7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:58.905315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-abe0c9cd-1588-41ba-88bd-2838af5f0fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:58.922104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0c9cd-1588-41ba-88bd-2838af5f0fd7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:58.927577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:58.928458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:58.929005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:58.929238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] PATH_INFO: `/v3/projects/9d89beafb10246f7b9087ca7cd75f239` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:58.930853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d89beafb10246f7b9087ca7cd75f239)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:58.934132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:59.006227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:59.009142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d89beafb10246f7b9087ca7cd75f239', 'request_id': 'req-04ceb758-ec83-45b0-934b-9e25b4f114bc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:59.035454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d89beafb10246f7b9087ca7cd75f239', 'request_id': 'req-04ceb758-ec83-45b0-934b-9e25b4f114bc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:59.053946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d89beafb10246f7b9087ca7cd75f239', 'request_id': 'req-04ceb758-ec83-45b0-934b-9e25b4f114bc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:59.087712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] The token cache is being invalidate because project 9d89beafb10246f7b9087ca7cd75f239 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:24:59.088239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:59.089262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-04ceb758-ec83-45b0-934b-9e25b4f114bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:24:59.093742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1164/2325] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:24:58 2022] DELETE /identity/v3/projects/9d89beafb10246f7b9087ca7cd75f239 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:59.101144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5465dd-eff9-41d7-b7c5-0da70ba0d3ee None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.101590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5465dd-eff9-41d7-b7c5-0da70ba0d3ee None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.101868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5465dd-eff9-41d7-b7c5-0da70ba0d3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.128433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4d5465dd-eff9-41d7-b7c5-0da70ba0d3ee None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:59.129821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4d5465dd-eff9-41d7-b7c5-0da70ba0d3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:59.253490 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1162/2326] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:24:59.263941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e0bd-20d2-4f08-b036-7226bdd39435 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.264236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e0bd-20d2-4f08-b036-7226bdd39435 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.264521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e0bd-20d2-4f08-b036-7226bdd39435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.287926 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-75c8e0bd-20d2-4f08-b036-7226bdd39435 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:59.289910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-75c8e0bd-20d2-4f08-b036-7226bdd39435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:59.305867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1165/2327] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:59.311273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5465dd-eff9-41d7-b7c5-0da70ba0d3ee None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:59.327201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5465dd-eff9-41d7-b7c5-0da70ba0d3ee None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:59.335202 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:59.336101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.336734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.336734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.337379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:24:59.338499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:59.340405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:59.360491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1163/2328] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:24:59.365626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e0bd-20d2-4f08-b036-7226bdd39435 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:59.378327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e0bd-20d2-4f08-b036-7226bdd39435 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:59.385994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dab299-3779-43ca-8f63-a6e47475226f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:59.387434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab299-3779-43ca-8f63-a6e47475226f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.387868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab299-3779-43ca-8f63-a6e47475226f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.388289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab299-3779-43ca-8f63-a6e47475226f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.389402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dab299-3779-43ca-8f63-a6e47475226f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:59.391940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dab299-3779-43ca-8f63-a6e47475226f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:59.452680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1166/2329] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:24:59 2022] POST /identity/v3/projects => generated 383 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:59.457425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:59.479161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecf2bb-05f0-497f-bcbd-3f8dbb8915f6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:59.532465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:24:59.594636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb9850-3413-4e1e-bd8d-896735a0c5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:59.595542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb9850-3413-4e1e-bd8d-896735a0c5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.595841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb9850-3413-4e1e-bd8d-896735a0c5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.596188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb9850-3413-4e1e-bd8d-896735a0c5a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.597151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb9850-3413-4e1e-bd8d-896735a0c5a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:59.599442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb9850-3413-4e1e-bd8d-896735a0c5a6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:59.631657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1164/2330] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:24:59 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:24:59.637000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dab299-3779-43ca-8f63-a6e47475226f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:59.650863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dab299-3779-43ca-8f63-a6e47475226f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:59.657822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ebee-efbe-4037-b18a-3d3e292ec807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:59.658740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ebee-efbe-4037-b18a-3d3e292ec807 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.659033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ebee-efbe-4037-b18a-3d3e292ec807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.659315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f905ebee-efbe-4037-b18a-3d3e292ec807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.660767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905ebee-efbe-4037-b18a-3d3e292ec807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:59.663343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f905ebee-efbe-4037-b18a-3d3e292ec807 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:59.695698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1167/2331] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:59.703485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb9850-3413-4e1e-bd8d-896735a0c5a6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:59.714807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb9850-3413-4e1e-bd8d-896735a0c5a6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:59.720725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34045b6e-5c56-44dd-90e3-40ca5985bff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:59.721955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045b6e-5c56-44dd-90e3-40ca5985bff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.722279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045b6e-5c56-44dd-90e3-40ca5985bff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.722609 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34045b6e-5c56-44dd-90e3-40ca5985bff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.723386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34045b6e-5c56-44dd-90e3-40ca5985bff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:59.725029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34045b6e-5c56-44dd-90e3-40ca5985bff8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:59.746954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1165/2332] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:24:59.747166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ebee-efbe-4037-b18a-3d3e292ec807 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:59.760420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f905ebee-efbe-4037-b18a-3d3e292ec807 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:24:59.769810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:24:59.771490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.771861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.773239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] PATH_INFO: `/v3/projects/546929b1810a4d20b273f3e5492ec6ed/users/71df16d1e792468585702fd688b8d3f2/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.791347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=546929b1810a4d20b273f3e5492ec6ed, user_id=71df16d1e792468585702fd688b8d3f2, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:24:59.794601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:24:59.813227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1168/2333] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:24:59 2022] PUT /identity/v3/projects/546929b1810a4d20b273f3e5492ec6ed/users/71df16d1e792468585702fd688b8d3f2/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:24:59.819528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd348d-d560-4b37-867b-2446f980ef55 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:24:59.819528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd348d-d560-4b37-867b-2446f980ef55 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:24:59.822358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd348d-d560-4b37-867b-2446f980ef55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:24:59.847037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-36fd348d-d560-4b37-867b-2446f980ef55 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:24:59.849946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-36fd348d-d560-4b37-867b-2446f980ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:24:59.958168 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1166/2334] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:24:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:24:59.973594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:24:59.975806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:00.064870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51986e-1b74-4535-8974-438ea6ba21b8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:00.071974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef049b5-cd40-4754-a140-94a268877cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:00.073001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef049b5-cd40-4754-a140-94a268877cef service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:00.073397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef049b5-cd40-4754-a140-94a268877cef service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:00.073722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef049b5-cd40-4754-a140-94a268877cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:00.095028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef049b5-cd40-4754-a140-94a268877cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:00.096938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef049b5-cd40-4754-a140-94a268877cef service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:00.113058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1169/2335] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:24:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:05.031019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd348d-d560-4b37-867b-2446f980ef55 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:05.033630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-36fd348d-d560-4b37-867b-2446f980ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:05.051175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd348d-d560-4b37-867b-2446f980ef55 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:05.058703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:05.060274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:05.061376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:05.062185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] PATH_INFO: `/v3/users/934d71e4968144ae9c0a885d2ab39fbc` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:05.064464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] RBAC: Authorizing `identity:delete_user(user_id=934d71e4968144ae9c0a885d2ab39fbc)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:05.066803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:05.143243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934d71e4968144ae9c0a885d2ab39fbc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:05.162209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934d71e4968144ae9c0a885d2ab39fbc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:05.163338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934d71e4968144ae9c0a885d2ab39fbc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:05.174023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934d71e4968144ae9c0a885d2ab39fbc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:05.184316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934d71e4968144ae9c0a885d2ab39fbc'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:05.193920 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1167/2336] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:25:05 2022] DELETE /identity/v3/users/934d71e4968144ae9c0a885d2ab39fbc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:25:05.204096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6d611-3996-4331-b9f1-49a3357dde16 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:05.204872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6d611-3996-4331-b9f1-49a3357dde16 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:05.205994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6d611-3996-4331-b9f1-49a3357dde16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:05.235303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-99d6d611-3996-4331-b9f1-49a3357dde16 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:05.237754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99d6d611-3996-4331-b9f1-49a3357dde16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:05.349086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1170/2337] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:05.362125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:05.363988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:05.461432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:05.506979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9eb949-1753-411c-ac28-72930e29233c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:05.511921 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e7dc2-7db0-4315-a99f-1c294fca2f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:05.518530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e7dc2-7db0-4315-a99f-1c294fca2f26 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:05.518530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e7dc2-7db0-4315-a99f-1c294fca2f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:05.518530 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e7dc2-7db0-4315-a99f-1c294fca2f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:05.546792 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e7dc2-7db0-4315-a99f-1c294fca2f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:05.548625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e7dc2-7db0-4315-a99f-1c294fca2f26 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:05.595355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1168/2338] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:06.330068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6d611-3996-4331-b9f1-49a3357dde16 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:06.332578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-99d6d611-3996-4331-b9f1-49a3357dde16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:06.366103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6d611-3996-4331-b9f1-49a3357dde16 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:06.384803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:06.387434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:06.387771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:06.388065 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] PATH_INFO: `/v3/projects/3621d57747c544a8aa4e683140cd9eaa` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:06.392259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3621d57747c544a8aa4e683140cd9eaa)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:06.400000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:06.416054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:06.484283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3621d57747c544a8aa4e683140cd9eaa', 'request_id': 'req-84f9ec39-6135-4428-85ab-21adb3a3f366'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:06.491495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3621d57747c544a8aa4e683140cd9eaa', 'request_id': 'req-84f9ec39-6135-4428-85ab-21adb3a3f366'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:06.520088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3621d57747c544a8aa4e683140cd9eaa', 'request_id': 'req-84f9ec39-6135-4428-85ab-21adb3a3f366'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:06.540946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] The token cache is being invalidate because project 3621d57747c544a8aa4e683140cd9eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:25:06.541573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:06.542553 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84f9ec39-6135-4428-85ab-21adb3a3f366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:06.545265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1171/2339] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:25:06 2022] DELETE /identity/v3/projects/3621d57747c544a8aa4e683140cd9eaa => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:25:06.552319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b11e3-215d-4a02-b261-06536601e6ca None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:06.553859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b11e3-215d-4a02-b261-06536601e6ca None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:06.554463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b11e3-215d-4a02-b261-06536601e6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:06.606260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-608b11e3-215d-4a02-b261-06536601e6ca None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:06.611917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-608b11e3-215d-4a02-b261-06536601e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:06.782732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc4e3-4a07-48cd-a1c9-417ab04f5ef5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:06.783365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc4e3-4a07-48cd-a1c9-417ab04f5ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:06.783825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dc4e3-4a07-48cd-a1c9-417ab04f5ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:06.786064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1169/2340] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:25:06.818314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-925dc4e3-4a07-48cd-a1c9-417ab04f5ef5 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:06.820109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-925dc4e3-4a07-48cd-a1c9-417ab04f5ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:06.833101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1172/2341] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:25:06.837733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b11e3-215d-4a02-b261-06536601e6ca None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:06.856747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608b11e3-215d-4a02-b261-06536601e6ca None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:06.866689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:06.867644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:06.868106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:06.868561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:06.869671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:25:06.870220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:06.872234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:06.906511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1170/2342] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 17:25:06.911586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc4e3-4a07-48cd-a1c9-417ab04f5ef5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:06.927551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dc4e3-4a07-48cd-a1c9-417ab04f5ef5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:06.970704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d32f12-455e-4995-b8e3-7406bbf403cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:06.972401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32f12-455e-4995-b8e3-7406bbf403cb admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:06.972973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32f12-455e-4995-b8e3-7406bbf403cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:06.973631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d32f12-455e-4995-b8e3-7406bbf403cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:06.974723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d32f12-455e-4995-b8e3-7406bbf403cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:06.978433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d32f12-455e-4995-b8e3-7406bbf403cb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:07.042934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1173/2343] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:25:06 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:07.045290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:07.074663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5bb75-1bde-4352-aff2-d542a2f5e905 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:07.182348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b9d1c-39e2-44a2-8586-12639372bb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:07.184468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b9d1c-39e2-44a2-8586-12639372bb81 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:07.185265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b9d1c-39e2-44a2-8586-12639372bb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:07.186082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b9d1c-39e2-44a2-8586-12639372bb81 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:07.187253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b9d1c-39e2-44a2-8586-12639372bb81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:07.190295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b9d1c-39e2-44a2-8586-12639372bb81 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:07.252627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1171/2344] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:25:07 2022] POST /identity/v3/users => generated 286 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:07.254273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d32f12-455e-4995-b8e3-7406bbf403cb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:07.271617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d32f12-455e-4995-b8e3-7406bbf403cb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:07.287855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7096a-b499-4d07-811e-4b1eba2defae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:07.289023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7096a-b499-4d07-811e-4b1eba2defae admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:07.289573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7096a-b499-4d07-811e-4b1eba2defae admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:07.290095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7096a-b499-4d07-811e-4b1eba2defae admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:07.292961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7096a-b499-4d07-811e-4b1eba2defae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:07.303006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f7096a-b499-4d07-811e-4b1eba2defae admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:07.332075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1174/2345] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:25:07.333010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b9d1c-39e2-44a2-8586-12639372bb81 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:07.360954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b9d1c-39e2-44a2-8586-12639372bb81 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:07.368991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44902773-b0c4-4527-aff3-bafa481f51fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:07.369915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44902773-b0c4-4527-aff3-bafa481f51fb admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:07.370367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44902773-b0c4-4527-aff3-bafa481f51fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:07.370847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44902773-b0c4-4527-aff3-bafa481f51fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:07.371838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44902773-b0c4-4527-aff3-bafa481f51fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:07.374374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44902773-b0c4-4527-aff3-bafa481f51fb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:07.395511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1172/2346] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:25:07.400088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7096a-b499-4d07-811e-4b1eba2defae admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:07.416037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f7096a-b499-4d07-811e-4b1eba2defae admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:07.423318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd486a-5c58-4243-a08f-eb8a9dc190ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:07.425636 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd486a-5c58-4243-a08f-eb8a9dc190ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:07.426420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd486a-5c58-4243-a08f-eb8a9dc190ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:07.427233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd486a-5c58-4243-a08f-eb8a9dc190ae admin admin] PATH_INFO: `/v3/projects/9f06cf783b2e4633a1e59bce930257bd/users/d143f1b1d27d40d5a0bf134fafef16e7/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:07.465979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd486a-5c58-4243-a08f-eb8a9dc190ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f06cf783b2e4633a1e59bce930257bd, user_id=d143f1b1d27d40d5a0bf134fafef16e7, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:07.470730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd486a-5c58-4243-a08f-eb8a9dc190ae admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:07.498871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1175/2347] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:25:07 2022] PUT /identity/v3/projects/9f06cf783b2e4633a1e59bce930257bd/users/d143f1b1d27d40d5a0bf134fafef16e7/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:25:07.512151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9a46-85bf-4dbe-98b0-20ad71f471ab None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:07.512499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9a46-85bf-4dbe-98b0-20ad71f471ab None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:07.512835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20c9a46-85bf-4dbe-98b0-20ad71f471ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:07.587846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e20c9a46-85bf-4dbe-98b0-20ad71f471ab None None] MFA Rules not processed for user `d143f1b1d27d40d5a0bf134fafef16e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:07.590482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e20c9a46-85bf-4dbe-98b0-20ad71f471ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:07.622350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:07.640150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6858e-3416-4908-848d-885a925eef20 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:07.640612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6858e-3416-4908-848d-885a925eef20 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:07.641240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6858e-3416-4908-848d-885a925eef20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:07.707510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-54a6858e-3416-4908-848d-885a925eef20 None None] MFA Rules not processed for user `71df16d1e792468585702fd688b8d3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:07.709302 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-54a6858e-3416-4908-848d-885a925eef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:07.779194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1173/2348] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:07.793662 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c9a46-85bf-4dbe-98b0-20ad71f471ab None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:07.798099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e20c9a46-85bf-4dbe-98b0-20ad71f471ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:07.823388 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1176/2349] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:25:07.828081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6858e-3416-4908-848d-885a925eef20 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:07.845066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-54a6858e-3416-4908-848d-885a925eef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:07.862134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:07.915038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20c9a46-85bf-4dbe-98b0-20ad71f471ab None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:07.919874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3f932-9e6b-4bd2-b03d-71efcb9da9ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:07.920275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f932-9e6b-4bd2-b03d-71efcb9da9ba service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:07.920558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f932-9e6b-4bd2-b03d-71efcb9da9ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:07.920825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3f932-9e6b-4bd2-b03d-71efcb9da9ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:07.928072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6858e-3416-4908-848d-885a925eef20 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:07.932274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3f932-9e6b-4bd2-b03d-71efcb9da9ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:07.933519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3f932-9e6b-4bd2-b03d-71efcb9da9ba service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:07.935090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ed35e-bd57-4ae6-8e7f-7a75ddb710a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:07.935856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ed35e-bd57-4ae6-8e7f-7a75ddb710a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:07.936638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ed35e-bd57-4ae6-8e7f-7a75ddb710a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:07.936638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ed35e-bd57-4ae6-8e7f-7a75ddb710a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:07.937369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ed35e-bd57-4ae6-8e7f-7a75ddb710a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:07.939373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ed35e-bd57-4ae6-8e7f-7a75ddb710a3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:07.945225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1174/2350] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:07.976788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1177/2351] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:25:07 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:07.981135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ed35e-bd57-4ae6-8e7f-7a75ddb710a3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:07.984222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3f932-9e6b-4bd2-b03d-71efcb9da9ba service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:08.020671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3f932-9e6b-4bd2-b03d-71efcb9da9ba service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:08.113058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:08.114297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:08.114645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:08.114961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] PATH_INFO: `/v3/users/d143f1b1d27d40d5a0bf134fafef16e7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:08.116432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d143f1b1d27d40d5a0bf134fafef16e7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:08.118775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:08.180125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ed35e-bd57-4ae6-8e7f-7a75ddb710a3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:08.197615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0ee67-5bb9-4c56-8657-8807a88b5ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:08.198447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0ee67-5bb9-4c56-8657-8807a88b5ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:08.198819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0ee67-5bb9-4c56-8657-8807a88b5ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:08.199126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0ee67-5bb9-4c56-8657-8807a88b5ff2 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:08.203241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0ee67-5bb9-4c56-8657-8807a88b5ff2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:08.210037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0ee67-5bb9-4c56-8657-8807a88b5ff2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:08.227373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd143f1b1d27d40d5a0bf134fafef16e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:08.265245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1178/2352] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:25:07 2022] POST /identity/v3/users => generated 299 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:08.269961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0ee67-5bb9-4c56-8657-8807a88b5ff2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:08.274470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:08.279895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd143f1b1d27d40d5a0bf134fafef16e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:08.281270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd143f1b1d27d40d5a0bf134fafef16e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:08.305351 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0ee67-5bb9-4c56-8657-8807a88b5ff2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:08.311341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd143f1b1d27d40d5a0bf134fafef16e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:08.317984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:08.320462 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:08.320673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:08.320983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:08.322960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:08.326504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:08.349267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1179/2353] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:25:08.350315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:08.353611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:08.369705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1030124-47a2-4311-860f-8d49f59b146d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:08.375310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea16030-ecac-40d7-9907-1c653727bb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:08.376360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea16030-ecac-40d7-9907-1c653727bb53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:08.376658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea16030-ecac-40d7-9907-1c653727bb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:08.376999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea16030-ecac-40d7-9907-1c653727bb53 admin admin] PATH_INFO: `/v3/projects/848c7ff5a10a47c0b98325add4101a73/users/5071d8cc509549dd98077dbf59591d35/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:08.404464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea16030-ecac-40d7-9907-1c653727bb53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=848c7ff5a10a47c0b98325add4101a73, user_id=5071d8cc509549dd98077dbf59591d35, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:08.409297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea16030-ecac-40d7-9907-1c653727bb53 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:08.553048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd143f1b1d27d40d5a0bf134fafef16e7'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:08.568899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1180/2354] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:25:08 2022] PUT /identity/v3/projects/848c7ff5a10a47c0b98325add4101a73/users/5071d8cc509549dd98077dbf59591d35/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:25:08.569166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1175/2355] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:25:07 2022] DELETE /identity/v3/users/d143f1b1d27d40d5a0bf134fafef16e7 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:25:08.574194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6143a-b21b-423e-b7b3-911868c6ef38 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:08.574490 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6143a-b21b-423e-b7b3-911868c6ef38 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:08.574747 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6143a-b21b-423e-b7b3-911868c6ef38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:08.575502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:08.619370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4ef6143a-b21b-423e-b7b3-911868c6ef38 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:08.626514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4ef6143a-b21b-423e-b7b3-911868c6ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:08.734174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0de87f-146e-4b1d-9f50-3b789f800adb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:08.748763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:08.753461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f6a63-bf23-4ad1-9731-c1f8a5bfc609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:08.754396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6a63-bf23-4ad1-9731-c1f8a5bfc609 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:08.754702 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6a63-bf23-4ad1-9731-c1f8a5bfc609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:08.755528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6a63-bf23-4ad1-9731-c1f8a5bfc609 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:08.756444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f6a63-bf23-4ad1-9731-c1f8a5bfc609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:08.760467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f6a63-bf23-4ad1-9731-c1f8a5bfc609 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:08.768210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1181/2356] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:08.781272 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6143a-b21b-423e-b7b3-911868c6ef38 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:08.785657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4ef6143a-b21b-423e-b7b3-911868c6ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:08.798068 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1176/2357] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:25:08.802804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f6a63-bf23-4ad1-9731-c1f8a5bfc609 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:08.829125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f6a63-bf23-4ad1-9731-c1f8a5bfc609 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:08.843747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8851a09-b940-40f9-a2d7-308467c1d4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:08.844843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8851a09-b940-40f9-a2d7-308467c1d4ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:08.845218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8851a09-b940-40f9-a2d7-308467c1d4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:08.851724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8851a09-b940-40f9-a2d7-308467c1d4ec admin admin] PATH_INFO: `/v3/projects/848c7ff5a10a47c0b98325add4101a73/users/5071d8cc509549dd98077dbf59591d35/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:08.861247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8851a09-b940-40f9-a2d7-308467c1d4ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=848c7ff5a10a47c0b98325add4101a73, user_id=5071d8cc509549dd98077dbf59591d35, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:08.864713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8851a09-b940-40f9-a2d7-308467c1d4ec admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:08.916298 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1177/2358] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:25:08 2022] PUT /identity/v3/projects/848c7ff5a10a47c0b98325add4101a73/users/5071d8cc509549dd98077dbf59591d35/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:25:08.917974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2acaa3-9c0e-4f5c-aeee-e0a80b308ce6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:08.920730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2acaa3-9c0e-4f5c-aeee-e0a80b308ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:08.921708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2acaa3-9c0e-4f5c-aeee-e0a80b308ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:08.954668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6143a-b21b-423e-b7b3-911868c6ef38 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:09.002779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9f2acaa3-9c0e-4f5c-aeee-e0a80b308ce6 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:09.004163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9f2acaa3-9c0e-4f5c-aeee-e0a80b308ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:09.111683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:09.112706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:09.113098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:09.113586 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:09.189129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1178/2359] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:25:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:09.192101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2acaa3-9c0e-4f5c-aeee-e0a80b308ce6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:09.192101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9f2acaa3-9c0e-4f5c-aeee-e0a80b308ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:09.263576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:09.266742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:09.333587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1182/2360] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:25:09.347755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:09.358871 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2acaa3-9c0e-4f5c-aeee-e0a80b308ce6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:09.367779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3a57-b8bf-446a-8c18-e01394b7db08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:09.369075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3a57-b8bf-446a-8c18-e01394b7db08 service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:09.370297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3a57-b8bf-446a-8c18-e01394b7db08 service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:09.370635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3a57-b8bf-446a-8c18-e01394b7db08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:09.384655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb3a57-b8bf-446a-8c18-e01394b7db08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:09.387341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb3a57-b8bf-446a-8c18-e01394b7db08 service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:09.402351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1179/2361] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:25:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:25:10.390862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:10.392683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:10.424326 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9dd7f7-c3d9-4d5f-ac57-cc62b9f0dc0e service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:10.439773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:10.440733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:10.441084 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:10.441479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] PATH_INFO: `/v3/projects/9f06cf783b2e4633a1e59bce930257bd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:10.443710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f06cf783b2e4633a1e59bce930257bd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:10.455549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:10.562758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f06cf783b2e4633a1e59bce930257bd', 'request_id': 'req-add4b4fd-3980-4647-933b-336ca5875d98'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:10.577824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f06cf783b2e4633a1e59bce930257bd', 'request_id': 'req-add4b4fd-3980-4647-933b-336ca5875d98'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:10.587848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:10.605260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f06cf783b2e4633a1e59bce930257bd', 'request_id': 'req-add4b4fd-3980-4647-933b-336ca5875d98'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:10.631161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] The token cache is being invalidate because project 9f06cf783b2e4633a1e59bce930257bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:25:10.631658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:10.632610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-add4b4fd-3980-4647-933b-336ca5875d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:25:10.637578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1183/2362] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:25:10 2022] DELETE /identity/v3/projects/9f06cf783b2e4633a1e59bce930257bd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:25:10.643269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158745-e880-4e59-bb16-b85505260625 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:10.643808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158745-e880-4e59-bb16-b85505260625 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:10.643808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de158745-e880-4e59-bb16-b85505260625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:10.699036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-de158745-e880-4e59-bb16-b85505260625 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:10.703372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-de158745-e880-4e59-bb16-b85505260625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:10.995909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1180/2363] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:11.012024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631394e-e50b-4d18-93a9-07fc602f9f86 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:11.012566 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631394e-e50b-4d18-93a9-07fc602f9f86 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:11.013070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631394e-e50b-4d18-93a9-07fc602f9f86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:11.087231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a631394e-e50b-4d18-93a9-07fc602f9f86 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:11.089385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a631394e-e50b-4d18-93a9-07fc602f9f86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:11.130416 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1184/2364] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:11.136889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158745-e880-4e59-bb16-b85505260625 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:11.173738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de158745-e880-4e59-bb16-b85505260625 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:11.206285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:11.207794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:11.209438 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:11.214890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:11.214890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:25:11.214890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:11.233561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:11.321762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1181/2365] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:25:11.327722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631394e-e50b-4d18-93a9-07fc602f9f86 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:11.351696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631394e-e50b-4d18-93a9-07fc602f9f86 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:11.361740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8824c-a28a-4831-a7ac-2af57aa12d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:11.364271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8824c-a28a-4831-a7ac-2af57aa12d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:11.364923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8824c-a28a-4831-a7ac-2af57aa12d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:11.365630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8824c-a28a-4831-a7ac-2af57aa12d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:11.368015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8824c-a28a-4831-a7ac-2af57aa12d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:11.375336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8824c-a28a-4831-a7ac-2af57aa12d82 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:11.467316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1185/2366] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:25:11 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:11.474682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:11.522886 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17bb9c-1c58-4772-84a6-d317970a74f5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:11.670296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45960148-f3e5-40fe-bc69-1f2a108bea6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:11.673656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45960148-f3e5-40fe-bc69-1f2a108bea6b admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:11.674166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45960148-f3e5-40fe-bc69-1f2a108bea6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:11.674598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45960148-f3e5-40fe-bc69-1f2a108bea6b admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:11.675498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45960148-f3e5-40fe-bc69-1f2a108bea6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:11.678150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45960148-f3e5-40fe-bc69-1f2a108bea6b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:11.714891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1182/2367] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:25:11 2022] POST /identity/v3/users => generated 290 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:11.719428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8824c-a28a-4831-a7ac-2af57aa12d82 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:11.731198 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8824c-a28a-4831-a7ac-2af57aa12d82 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:11.739148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79376a30-70b8-4b2d-bdd0-6c1e6d3eda81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:11.740395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79376a30-70b8-4b2d-bdd0-6c1e6d3eda81 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:11.740876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79376a30-70b8-4b2d-bdd0-6c1e6d3eda81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:11.741420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79376a30-70b8-4b2d-bdd0-6c1e6d3eda81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:11.742663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79376a30-70b8-4b2d-bdd0-6c1e6d3eda81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:11.745126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79376a30-70b8-4b2d-bdd0-6c1e6d3eda81 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:11.768764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1186/2368] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:25:11.773405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45960148-f3e5-40fe-bc69-1f2a108bea6b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:11.792904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45960148-f3e5-40fe-bc69-1f2a108bea6b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:11.813478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93934281-5d07-45b8-b851-212be5608246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:11.813822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934281-5d07-45b8-b851-212be5608246 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:11.814196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934281-5d07-45b8-b851-212be5608246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:11.815386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934281-5d07-45b8-b851-212be5608246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:11.817039 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93934281-5d07-45b8-b851-212be5608246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:11.839264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93934281-5d07-45b8-b851-212be5608246 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:11.878687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1183/2369] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:25:11.883183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79376a30-70b8-4b2d-bdd0-6c1e6d3eda81 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:11.897464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79376a30-70b8-4b2d-bdd0-6c1e6d3eda81 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:11.905470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:11.906463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:11.906714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:11.907603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] PATH_INFO: `/v3/projects/bd1488226a424c6e9e7f94f31b020e4f/users/fbbafeb56a4d4fd282275e4198ede783/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:11.932473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1488226a424c6e9e7f94f31b020e4f, user_id=fbbafeb56a4d4fd282275e4198ede783, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:11.935985 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:11.973968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1187/2370] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:25:11 2022] PUT /identity/v3/projects/bd1488226a424c6e9e7f94f31b020e4f/users/fbbafeb56a4d4fd282275e4198ede783/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:25:11.980112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90352882-ab85-43a7-90bc-ca7dc1cebcb4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:11.980425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90352882-ab85-43a7-90bc-ca7dc1cebcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:11.984552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90352882-ab85-43a7-90bc-ca7dc1cebcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:12.006041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-90352882-ab85-43a7-90bc-ca7dc1cebcb4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:12.007509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-90352882-ab85-43a7-90bc-ca7dc1cebcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:12.090893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:12.123855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1184/2371] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:25:12.130772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:12.133383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:12.280602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f9ed8-1195-4b97-b59c-e1e21195ce1e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:12.285407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b19e9-2cd6-4301-bd1a-0c4d676089d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:12.286219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b19e9-2cd6-4301-bd1a-0c4d676089d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:12.287783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b19e9-2cd6-4301-bd1a-0c4d676089d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:12.288105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b19e9-2cd6-4301-bd1a-0c4d676089d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:12.305160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b19e9-2cd6-4301-bd1a-0c4d676089d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:12.308880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b19e9-2cd6-4301-bd1a-0c4d676089d8 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:12.328225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1188/2372] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:22.106617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:22.318033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=1,keystone:SELECT=57,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:25.176322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248419-37d3-4f16-94fc-d334913ce449 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:25.176814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248419-37d3-4f16-94fc-d334913ce449 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:25.176895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93248419-37d3-4f16-94fc-d334913ce449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:25.295621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-93248419-37d3-4f16-94fc-d334913ce449 None None] MFA Rules not processed for user `5071d8cc509549dd98077dbf59591d35`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:25.302847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-93248419-37d3-4f16-94fc-d334913ce449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:25.373792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394cd66-b7fd-4f92-a5d0-829cb7e48c4a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:25.374579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394cd66-b7fd-4f92-a5d0-829cb7e48c4a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:25.374579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394cd66-b7fd-4f92-a5d0-829cb7e48c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:25.473603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b394cd66-b7fd-4f92-a5d0-829cb7e48c4a None None] MFA Rules not processed for user `fbbafeb56a4d4fd282275e4198ede783`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:25.475507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b394cd66-b7fd-4f92-a5d0-829cb7e48c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:25.585310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1185/2373] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:25.682212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1189/2374] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:25.683890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93248419-37d3-4f16-94fc-d334913ce449 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:25.701987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93248419-37d3-4f16-94fc-d334913ce449 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:25.707979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae9229-5c91-4386-8edb-0b503a3eeae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:25.708670 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae9229-5c91-4386-8edb-0b503a3eeae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:25.709005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae9229-5c91-4386-8edb-0b503a3eeae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:25.709364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae9229-5c91-4386-8edb-0b503a3eeae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:25.710070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae9229-5c91-4386-8edb-0b503a3eeae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:25.713996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae9229-5c91-4386-8edb-0b503a3eeae4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:25.767847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1186/2375] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:25:25 2022] POST /identity/v3/projects => generated 379 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:25.771058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394cd66-b7fd-4f92-a5d0-829cb7e48c4a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:25.799993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394cd66-b7fd-4f92-a5d0-829cb7e48c4a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:25.930066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbc87c-62e3-478c-aa89-b1a7d2446c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:25.931203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbc87c-62e3-478c-aa89-b1a7d2446c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:25.931655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbc87c-62e3-478c-aa89-b1a7d2446c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:25.932010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbc87c-62e3-478c-aa89-b1a7d2446c96 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:25.932815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbc87c-62e3-478c-aa89-b1a7d2446c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:25.938592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbc87c-62e3-478c-aa89-b1a7d2446c96 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:25.973936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1190/2376] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:25:25 2022] POST /identity/v3/users => generated 296 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:25:25.978146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae9229-5c91-4386-8edb-0b503a3eeae4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:25.995784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae9229-5c91-4386-8edb-0b503a3eeae4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:26.002514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fa3b6-13b8-4906-a103-f98e206ae94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:26.003338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fa3b6-13b8-4906-a103-f98e206ae94b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:26.003647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fa3b6-13b8-4906-a103-f98e206ae94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:26.003910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fa3b6-13b8-4906-a103-f98e206ae94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:26.004667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fa3b6-13b8-4906-a103-f98e206ae94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:26.006479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460fa3b6-13b8-4906-a103-f98e206ae94b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:26.045519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1187/2377] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:25:26.053083 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbc87c-62e3-478c-aa89-b1a7d2446c96 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:26.065444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbc87c-62e3-478c-aa89-b1a7d2446c96 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:26.071064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:26.072101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:26.072284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:26.072532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] PATH_INFO: `/v3/projects/6c0384215d0547d4b0e2a2228f06c84d/users/f1006a405d054de7bbc43c256561a151/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:26.099499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0384215d0547d4b0e2a2228f06c84d, user_id=f1006a405d054de7bbc43c256561a151, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:26.103866 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:26.128151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1191/2378] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:25:26 2022] PUT /identity/v3/projects/6c0384215d0547d4b0e2a2228f06c84d/users/f1006a405d054de7bbc43c256561a151/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:25:26.132776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fa3b6-13b8-4906-a103-f98e206ae94b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:26.160805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460fa3b6-13b8-4906-a103-f98e206ae94b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:26.199821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:26.202402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:26.391239 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22d986-20c4-4a8f-917d-898328e12bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:26.395985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22d986-20c4-4a8f-917d-898328e12bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:26.396433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22d986-20c4-4a8f-917d-898328e12bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:26.397481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22d986-20c4-4a8f-917d-898328e12bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:26.398498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22d986-20c4-4a8f-917d-898328e12bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:26.402363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22d986-20c4-4a8f-917d-898328e12bfa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:26.422210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc3857-ed0e-47e7-8428-75dfc1347ee9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:26.428626 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1188/2379] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:25:26.441401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22d986-20c4-4a8f-917d-898328e12bfa admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:26.443385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:26.448765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:26.449263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:26.449833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:26.453168 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:26.455253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22d986-20c4-4a8f-917d-898328e12bfa admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:26.467030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:26.471128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f14e4ed-f9af-4cba-ac42-ada1c24ed603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:26.472325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14e4ed-f9af-4cba-ac42-ada1c24ed603 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:26.473694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14e4ed-f9af-4cba-ac42-ada1c24ed603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:26.474323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14e4ed-f9af-4cba-ac42-ada1c24ed603 admin admin] PATH_INFO: `/v3/projects/6c0384215d0547d4b0e2a2228f06c84d/users/f1006a405d054de7bbc43c256561a151/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:26.477025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14e4ed-f9af-4cba-ac42-ada1c24ed603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0384215d0547d4b0e2a2228f06c84d, user_id=f1006a405d054de7bbc43c256561a151, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:26.481659 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f14e4ed-f9af-4cba-ac42-ada1c24ed603 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:26.504810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1189/2380] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:25:26 2022] PUT /identity/v3/projects/6c0384215d0547d4b0e2a2228f06c84d/users/f1006a405d054de7bbc43c256561a151/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:25:26.595178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:26.600162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:26.696928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1192/2381] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:27.211450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54762e5-5520-4b0b-bfa9-98dfbe01ca04 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:27.211850 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54762e5-5520-4b0b-bfa9-98dfbe01ca04 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:27.212024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54762e5-5520-4b0b-bfa9-98dfbe01ca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:27.258168 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b54762e5-5520-4b0b-bfa9-98dfbe01ca04 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:27.260251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b54762e5-5520-4b0b-bfa9-98dfbe01ca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:27.344025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:27.394808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1190/2382] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:25:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:27.414436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:27.416891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:27.572129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9d1de-d199-4c6a-a953-cc5d9d389a9d service glance] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:27.894683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:27.895707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:27.897476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:27.898160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:27.928875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:27.931924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:27.956248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1193/2383] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:25:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 17:25:37.331017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bac1f-1f64-4b9f-b99c-dac4eb3c9460 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:37.332037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bac1f-1f64-4b9f-b99c-dac4eb3c9460 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:37.332037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255bac1f-1f64-4b9f-b99c-dac4eb3c9460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:37.369242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-255bac1f-1f64-4b9f-b99c-dac4eb3c9460 None None] MFA Rules not processed for user `f1006a405d054de7bbc43c256561a151`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:37.370452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-255bac1f-1f64-4b9f-b99c-dac4eb3c9460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:37.469489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1191/2384] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:37 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:37.476781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:37.479197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:37.580156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:37.666946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876719b-71ea-4e32-be07-9478e07112dd service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:37.673016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c0489-91c5-489f-8f09-f862334f81ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:37.673744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c0489-91c5-489f-8f09-f862334f81ed service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:37.674034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c0489-91c5-489f-8f09-f862334f81ed service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:37.674412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c0489-91c5-489f-8f09-f862334f81ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:37.805061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c0489-91c5-489f-8f09-f862334f81ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:37.806708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c0489-91c5-489f-8f09-f862334f81ed service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:37.830789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1194/2385] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:40.434718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bac1f-1f64-4b9f-b99c-dac4eb3c9460 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:40.436199 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-255bac1f-1f64-4b9f-b99c-dac4eb3c9460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:40.762720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255bac1f-1f64-4b9f-b99c-dac4eb3c9460 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:40.774734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe38ce-faf4-4998-a3cc-a3a10867ca57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:40.775479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe38ce-faf4-4998-a3cc-a3a10867ca57 service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:40.778114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe38ce-faf4-4998-a3cc-a3a10867ca57 service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:40.778617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe38ce-faf4-4998-a3cc-a3a10867ca57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:40.783763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7fbe38ce-faf4-4998-a3cc-a3a10867ca57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:40.813254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe38ce-faf4-4998-a3cc-a3a10867ca57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:40.815681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe38ce-faf4-4998-a3cc-a3a10867ca57 service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:40.838958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1192/2386] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:25:43.196660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4a3e9-aedb-4e87-afa7-80afd9753e76 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:43.198172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4a3e9-aedb-4e87-afa7-80afd9753e76 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:43.198690 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c4a3e9-aedb-4e87-afa7-80afd9753e76 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:43.228396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1195/2387] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:25:43 2022] GET /identity => generated 270 bytes in 33 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:25:43.241054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cbb21-996a-459e-a8fd-09ebba84a76d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:43.242505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cbb21-996a-459e-a8fd-09ebba84a76d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:43.242949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909cbb21-996a-459e-a8fd-09ebba84a76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:43.342192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-909cbb21-996a-459e-a8fd-09ebba84a76d None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:43.343846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-909cbb21-996a-459e-a8fd-09ebba84a76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:43.373775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1193/2388] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:25:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:45.974226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4a3e9-aedb-4e87-afa7-80afd9753e76 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:46.041380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41645-f31b-471d-a1a6-8d436b8a4006 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:46.042007 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41645-f31b-471d-a1a6-8d436b8a4006 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:46.042397 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da41645-f31b-471d-a1a6-8d436b8a4006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:46.064382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c4a3e9-aedb-4e87-afa7-80afd9753e76 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:46.065714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4da41645-f31b-471d-a1a6-8d436b8a4006 None None] MFA Rules not processed for user `71df16d1e792468585702fd688b8d3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:46.067560 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4da41645-f31b-471d-a1a6-8d436b8a4006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:46.068626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c730a-4582-4d52-9397-7686dfbc14c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:46.069847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c730a-4582-4d52-9397-7686dfbc14c7 service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:46.069847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c730a-4582-4d52-9397-7686dfbc14c7 service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:46.070091 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c730a-4582-4d52-9397-7686dfbc14c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:46.072935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:46.084470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c730a-4582-4d52-9397-7686dfbc14c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:46.087028 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c730a-4582-4d52-9397-7686dfbc14c7 service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:46.105896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1196/2389] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:46.174941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1194/2390] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:25:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:25:46.207657 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba634dc5-95d4-4574-9679-517eb52300fa None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:46.208043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba634dc5-95d4-4574-9679-517eb52300fa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:46.208453 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba634dc5-95d4-4574-9679-517eb52300fa None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:46.209946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1197/2391] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:25:46.216827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1fe42-46a5-4054-abcc-05a6361514a7 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:46.217327 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1fe42-46a5-4054-abcc-05a6361514a7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:46.217634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1fe42-46a5-4054-abcc-05a6361514a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:46.241468 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-89b1fe42-46a5-4054-abcc-05a6361514a7 None None] MFA Rules not processed for user `72fb6996e1974060b162070f8da365aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:46.243680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-89b1fe42-46a5-4054-abcc-05a6361514a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:46.314315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1195/2392] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:25:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:46.328852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba634dc5-95d4-4574-9679-517eb52300fa None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:46.341660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba634dc5-95d4-4574-9679-517eb52300fa None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:46.346399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715a9f2-1656-4883-86f1-ccf371f2112f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:46.347148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715a9f2-1656-4883-86f1-ccf371f2112f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:46.347433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715a9f2-1656-4883-86f1-ccf371f2112f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:46.347707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715a9f2-1656-4883-86f1-ccf371f2112f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:46.356075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715a9f2-1656-4883-86f1-ccf371f2112f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:46.357562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715a9f2-1656-4883-86f1-ccf371f2112f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:46.366724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1198/2393] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:46.463650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b736-35fc-42c7-a505-970e223448f7 None None] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:46.463995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b736-35fc-42c7-a505-970e223448f7 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:46.464318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1b736-35fc-42c7-a505-970e223448f7 None None] PATH_INFO: `/` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:46.465860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1196/2394] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:25:46.472546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc464-85b0-4246-a9bc-6ba97c1cf151 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:46.473236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc464-85b0-4246-a9bc-6ba97c1cf151 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:46.473656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afccc464-85b0-4246-a9bc-6ba97c1cf151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:46.500753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-afccc464-85b0-4246-a9bc-6ba97c1cf151 None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:46.502420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-afccc464-85b0-4246-a9bc-6ba97c1cf151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:46.520153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1199/2395] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:25:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:25:46.537949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1b736-35fc-42c7-a505-970e223448f7 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:46.555313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1b736-35fc-42c7-a505-970e223448f7 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:46.559442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3651220-b575-4761-8e18-47b03f966485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:46.560458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3651220-b575-4761-8e18-47b03f966485 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:46.560458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3651220-b575-4761-8e18-47b03f966485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:46.560458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3651220-b575-4761-8e18-47b03f966485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:46.574738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3651220-b575-4761-8e18-47b03f966485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:46.576366 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3651220-b575-4761-8e18-47b03f966485 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:46.588294 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1197/2396] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:25:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:25:49.190908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3881-46d9-4bfb-b977-b40c7df0df90 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:49.192311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3881-46d9-4bfb-b977-b40c7df0df90 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:49.192597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b3881-46d9-4bfb-b977-b40c7df0df90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:49.259389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-907b3881-46d9-4bfb-b977-b40c7df0df90 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:25:49.260615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-907b3881-46d9-4bfb-b977-b40c7df0df90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:25:49.268964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:49.283582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1200/2397] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:25:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:25:49.297752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3651220-b575-4761-8e18-47b03f966485 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:25:49.326865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3651220-b575-4761-8e18-47b03f966485 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:25:49.336529 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987763f-bcbf-4113-8824-5cbd6da81b90 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:25:49.337514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987763f-bcbf-4113-8824-5cbd6da81b90 service swift] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:25:49.337931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987763f-bcbf-4113-8824-5cbd6da81b90 service swift] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:25:49.338336 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987763f-bcbf-4113-8824-5cbd6da81b90 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:25:49.352845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987763f-bcbf-4113-8824-5cbd6da81b90 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:25:49.355720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987763f-bcbf-4113-8824-5cbd6da81b90 service swift] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:25:49.371429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1198/2398] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:25:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:25:59.279992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:25:59.364445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=60 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:05.955659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3704d-2a6c-4efc-9d29-508abfca3582 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:05.956297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3704d-2a6c-4efc-9d29-508abfca3582 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:05.956297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3704d-2a6c-4efc-9d29-508abfca3582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:05.982653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-15c3704d-2a6c-4efc-9d29-508abfca3582 None None] MFA Rules not processed for user `fbbafeb56a4d4fd282275e4198ede783`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:05.984189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-15c3704d-2a6c-4efc-9d29-508abfca3582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:05.992367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1201/2399] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:26:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:06.270175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7802a-a201-431f-919a-26815d6af2c8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:06.270711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7802a-a201-431f-919a-26815d6af2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:06.270711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7802a-a201-431f-919a-26815d6af2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:06.292885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7ab7802a-a201-431f-919a-26815d6af2c8 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:06.294142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7ab7802a-a201-431f-919a-26815d6af2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:06.303631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1199/2400] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:26:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:06.331220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3704d-2a6c-4efc-9d29-508abfca3582 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:06.345074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3704d-2a6c-4efc-9d29-508abfca3582 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:06.350317 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:06.351151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:06.351542 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:06.351914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:06.361955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:06.363660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:06.373736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1202/2401] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:26:16.286388 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:16.369606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=9 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:33.101439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea9dcc-d641-4870-9c0d-84cce83c28ee None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:33.102225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea9dcc-d641-4870-9c0d-84cce83c28ee None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:33.102225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea9dcc-d641-4870-9c0d-84cce83c28ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:33.123462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-1cea9dcc-d641-4870-9c0d-84cce83c28ee None None] MFA Rules not processed for user `8b088ae5ce5046ae83d9ae151dc52eca`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:33.124719 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1cea9dcc-d641-4870-9c0d-84cce83c28ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:33.186884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1200/2402] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:26:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:34.209268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:34.211761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:34.237709 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e345a575-b832-455e-8c38-3c013fb8d4e8 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:34.247292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f74d8-3b2e-4835-9a00-181e9cab5bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:34.248990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f74d8-3b2e-4835-9a00-181e9cab5bfb service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:34.249633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f74d8-3b2e-4835-9a00-181e9cab5bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:34.250380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f74d8-3b2e-4835-9a00-181e9cab5bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:34.372508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f74d8-3b2e-4835-9a00-181e9cab5bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:34.375782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f74d8-3b2e-4835-9a00-181e9cab5bfb service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:34.404766 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1203/2403] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:36.338778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043e975-b544-4771-8fe0-08c07eb198e2 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:36.339736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043e975-b544-4771-8fe0-08c07eb198e2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:36.339736 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043e975-b544-4771-8fe0-08c07eb198e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:36.370739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-c043e975-b544-4771-8fe0-08c07eb198e2 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:36.372372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c043e975-b544-4771-8fe0-08c07eb198e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:36.384372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1201/2404] 10.222.0.66 () {58 vars in 967 bytes} [Thu Jun 16 17:26:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:36.403629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f74d8-3b2e-4835-9a00-181e9cab5bfb service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:36.431477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f74d8-3b2e-4835-9a00-181e9cab5bfb service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:36.438170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe0b2b-581a-4cbb-b996-0ddbd93fa211 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:36.439252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe0b2b-581a-4cbb-b996-0ddbd93fa211 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:36.439564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe0b2b-581a-4cbb-b996-0ddbd93fa211 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:36.440010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe0b2b-581a-4cbb-b996-0ddbd93fa211 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:36.463380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe0b2b-581a-4cbb-b996-0ddbd93fa211 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:36.465448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe0b2b-581a-4cbb-b996-0ddbd93fa211 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:36.493015 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1204/2405] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:26:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:26:40.181403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043e975-b544-4771-8fe0-08c07eb198e2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:26:40.197783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043e975-b544-4771-8fe0-08c07eb198e2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:26:40.202480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f458bb-fae9-4c92-afc2-cc031e249d61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:26:40.203480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f458bb-fae9-4c92-afc2-cc031e249d61 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:40.203757 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f458bb-fae9-4c92-afc2-cc031e249d61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:40.204255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f458bb-fae9-4c92-afc2-cc031e249d61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:40.216658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f458bb-fae9-4c92-afc2-cc031e249d61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:26:40.218121 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f458bb-fae9-4c92-afc2-cc031e249d61 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:26:40.244463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1202/2406] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:26:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:26:43.952854 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0848-78aa-4c53-a969-30ed0f9a7f05 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:26:43.954379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0848-78aa-4c53-a969-30ed0f9a7f05 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:26:43.955038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a0848-78aa-4c53-a969-30ed0f9a7f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:26:44.003265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-760a0848-78aa-4c53-a969-30ed0f9a7f05 None None] MFA Rules not processed for user `0cd8a309f6914d83b8e72bc60a5d7924`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:26:44.005964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-760a0848-78aa-4c53-a969-30ed0f9a7f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:26:44.150465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1205/2407] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:26:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:26:50.224526 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=34 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:26:54.128465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=61 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:27:40.248187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f458bb-fae9-4c92-afc2-cc031e249d61 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:40.259027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f458bb-fae9-4c92-afc2-cc031e249d61 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:40.264685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:40.265983 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:40.266340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:40.266630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:40.268725 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:40.282609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a0848-78aa-4c53-a969-30ed0f9a7f05 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:40.285328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:40.286870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:40.295155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a0848-78aa-4c53-a969-30ed0f9a7f05 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:40.300794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1203/2408] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:40.301069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:40.301864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:40.302486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:40.302792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:40.304974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:40.319624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:40.321291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:40.333644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1206/2409] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:27:42.761559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:42.788285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51515c-ff5a-45a1-8768-0654a386a57b service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:42.795595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:42.797216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:42.797721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:42.797875 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:42.804476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:42.815640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:42.817624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:42.895653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:42.897619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:42.908929 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1204/2410] 10.222.0.66 () {60 vars in 1396 bytes} [Thu Jun 16 17:27:42 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3256 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:27:42.923445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b488d-27ac-4fe7-8eb0-5ad747b24448 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:42.932353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:42.933430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:42.933710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:42.933992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] PATH_INFO: `/v3/users/8b088ae5ce5046ae83d9ae151dc52eca` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:42.935384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b088ae5ce5046ae83d9ae151dc52eca)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:42.937736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:43.008472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b088ae5ce5046ae83d9ae151dc52eca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.009556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b088ae5ce5046ae83d9ae151dc52eca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.010186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b088ae5ce5046ae83d9ae151dc52eca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.024992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b088ae5ce5046ae83d9ae151dc52eca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.035434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3571ddad-48dc-4cfe-a374-fe83596e964e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b088ae5ce5046ae83d9ae151dc52eca'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.049375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1207/2411] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:27:42 2022] DELETE /identity/v3/users/8b088ae5ce5046ae83d9ae151dc52eca => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:27:43.246382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:43.269901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1c2d6-9758-4a7e-8c51-c6cb680c5909 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:43.345042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:43.345936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:43.346347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:43.346682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] PATH_INFO: `/v3/projects/f96239e8e212410aac9df2d35293c665` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:43.348122 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96239e8e212410aac9df2d35293c665)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:43.350396 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:43.411561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96239e8e212410aac9df2d35293c665', 'request_id': 'req-aa41706a-f8bd-4f5f-b281-cf23e8962770'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.420399 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96239e8e212410aac9df2d35293c665', 'request_id': 'req-aa41706a-f8bd-4f5f-b281-cf23e8962770'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.427458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96239e8e212410aac9df2d35293c665', 'request_id': 'req-aa41706a-f8bd-4f5f-b281-cf23e8962770'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.439178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] The token cache is being invalidate because project f96239e8e212410aac9df2d35293c665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:27:43.439682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.440206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:27:43.441837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1205/2412] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:27:43 2022] DELETE /identity/v3/projects/f96239e8e212410aac9df2d35293c665 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:27:47.667162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a288-2798-463f-a010-748c576fbb22 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:47.667749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a288-2798-463f-a010-748c576fbb22 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:47.667749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20a288-2798-463f-a010-748c576fbb22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:47.690340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dc20a288-2798-463f-a010-748c576fbb22 None None] MFA Rules not processed for user `68da2fd51cc84b2b8d641e1e6ac17c72`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:27:47.691515 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dc20a288-2798-463f-a010-748c576fbb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:47.757419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1208/2413] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:27:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:27:48.081135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:48.082806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:48.102916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:27:48.150307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41706a-f8bd-4f5f-b281-cf23e8962770 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:48.154452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:48.155333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:48.155619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:48.155891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:48.157771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:48.169216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:48.170778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:48.180409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1206/2414] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:48.526588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20a288-2798-463f-a010-748c576fbb22 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:48.528327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dc20a288-2798-463f-a010-748c576fbb22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:48.542328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:27:48.619653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20a288-2798-463f-a010-748c576fbb22 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:48.625688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:48.626352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:48.626639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:48.627550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:48.629636 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:48.642872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:48.644482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:48.655615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1209/2415] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:50.439906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:50.441990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:50.551881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5de86-8011-4bd2-bccf-77831acf472d service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:50.557375 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:50.558063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:50.558348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:50.558627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:50.560572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:50.639517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:50.641452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:50.667759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1207/2416] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:52.215941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:52.226563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc395d-e510-4d4a-8f83-322ee755d049 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:52.231042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ff731-33cf-4a7e-a008-246de5c90c85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:52.231722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff731-33cf-4a7e-a008-246de5c90c85 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:52.233532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff731-33cf-4a7e-a008-246de5c90c85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:52.233532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff731-33cf-4a7e-a008-246de5c90c85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:52.234510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9e1ff731-33cf-4a7e-a008-246de5c90c85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:52.301778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ff731-33cf-4a7e-a008-246de5c90c85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:52.303329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ff731-33cf-4a7e-a008-246de5c90c85 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:52.314122 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1210/2417] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:27:58.780818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:27:58.791480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6034d-6062-4bb9-805a-d976df90b828 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:27:58.795223 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:27:58.795979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:27:58.796152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:27:58.796428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:27:58.798290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:27:58.809125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:27:58.810633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:27:58.819838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1208/2418] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:27:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:02.252833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c20e6e-ede8-423d-89b0-243f6b077ffe None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:02.253468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c20e6e-ede8-423d-89b0-243f6b077ffe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:02.253468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c20e6e-ede8-423d-89b0-243f6b077ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:02.271641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-50c20e6e-ede8-423d-89b0-243f6b077ffe None None] MFA Rules not processed for user `1a3418bd5d7f430ca84efd5c83e38fed`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:02.273259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-50c20e6e-ede8-423d-89b0-243f6b077ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:02.281034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1211/2419] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:02.764015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:02.765930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:02.780378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b169809a-8e0f-4ded-a49e-35863c484873 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:02.786589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:02.788274 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:02.788728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:02.789680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:02.792451 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:02.805846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:02.807368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:02.820805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1209/2420] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:07.326151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c20e6e-ede8-423d-89b0-243f6b077ffe None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:07.328905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-50c20e6e-ede8-423d-89b0-243f6b077ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:07.478374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c20e6e-ede8-423d-89b0-243f6b077ffe None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:07.487405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:07.487713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:07.488448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:07.488734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:07.491067 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:07.511834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:07.515183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:07.527269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1212/2421] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:08.580009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:08.584419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:08.624541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17132c-b5c4-4f88-b166-387fa9875bc8 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:08.640101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:08.640945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:08.641361 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:08.641731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:08.650812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:08.666656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:08.668338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:08.681972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1210/2422] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:15.052158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:15.066349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727ed80c-91ea-4c0d-8d75-22d74b4cf408 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:15.072058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:15.072865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:15.073228 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:15.073496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:15.078803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:15.094683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:15.156561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:15.158289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:15.170751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1213/2423] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:18.676607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:19.418085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:19.420145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:19.442179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac1b1e-8d00-400d-aae7-db08417dbc62 service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:19.448016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:19.449066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:19.449477 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:19.449854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] PATH_INFO: `/v3/users/0cd8a309f6914d83b8e72bc60a5d7924` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:19.451304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd8a309f6914d83b8e72bc60a5d7924)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:19.453392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:19.520635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd8a309f6914d83b8e72bc60a5d7924'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.524266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd8a309f6914d83b8e72bc60a5d7924'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.525170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd8a309f6914d83b8e72bc60a5d7924'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.532894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd8a309f6914d83b8e72bc60a5d7924'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.542727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1be9c6fc-4f48-4be1-b516-e797e0fae85a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd8a309f6914d83b8e72bc60a5d7924'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.550559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1211/2424] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:19 2022] DELETE /identity/v3/users/0cd8a309f6914d83b8e72bc60a5d7924 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:19.778681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:19.789919 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447072c1-bd39-4c3f-b0fa-35a72eedaf09 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:19.851780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:19.852945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:19.853261 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:19.853543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] PATH_INFO: `/v3/projects/ccf5c45783454fc5a7872490196f07e0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:19.854774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf5c45783454fc5a7872490196f07e0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:19.856749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:19.908001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf5c45783454fc5a7872490196f07e0', 'request_id': 'req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.915077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf5c45783454fc5a7872490196f07e0', 'request_id': 'req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.924260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf5c45783454fc5a7872490196f07e0', 'request_id': 'req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.936281 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] The token cache is being invalidate because project ccf5c45783454fc5a7872490196f07e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:19.936570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.937256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8c56d0ea-b1d3-40c7-8849-a8af55d87d02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:19.939048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1214/2425] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:19 2022] DELETE /identity/v3/projects/ccf5c45783454fc5a7872490196f07e0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:19.944823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b072d-fd97-4fc1-b678-058804c6fedd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:19.945331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b072d-fd97-4fc1-b678-058804c6fedd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:19.945498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b072d-fd97-4fc1-b678-058804c6fedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:19.964986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-cc3b072d-fd97-4fc1-b678-058804c6fedd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:19.966248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-cc3b072d-fd97-4fc1-b678-058804c6fedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:20.032262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1212/2426] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:20.040029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566a450-9328-4c52-bad5-f07e92f15711 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.040318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566a450-9328-4c52-bad5-f07e92f15711 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.040597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e566a450-9328-4c52-bad5-f07e92f15711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.062904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e566a450-9328-4c52-bad5-f07e92f15711 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:20.064113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e566a450-9328-4c52-bad5-f07e92f15711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:20.074055 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1215/2427] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:20.078403 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b072d-fd97-4fc1-b678-058804c6fedd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:20.088616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b072d-fd97-4fc1-b678-058804c6fedd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:20.093753 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747bc14-7468-4518-a640-1029627cface admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:20.094707 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747bc14-7468-4518-a640-1029627cface admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.094990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747bc14-7468-4518-a640-1029627cface admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.095268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747bc14-7468-4518-a640-1029627cface admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.096011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747bc14-7468-4518-a640-1029627cface admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:28:20.096321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747bc14-7468-4518-a640-1029627cface admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:20.097990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747bc14-7468-4518-a640-1029627cface admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:20.119394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1213/2428] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:28:20.120580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566a450-9328-4c52-bad5-f07e92f15711 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:20.130483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e566a450-9328-4c52-bad5-f07e92f15711 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:20.135651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeb666-68d3-4c5a-819a-48a04cab06fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:20.136338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeb666-68d3-4c5a-819a-48a04cab06fa admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.136620 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeb666-68d3-4c5a-819a-48a04cab06fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.136893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeb666-68d3-4c5a-819a-48a04cab06fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.137626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeb666-68d3-4c5a-819a-48a04cab06fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:20.139761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeb666-68d3-4c5a-819a-48a04cab06fa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:20.168149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1216/2429] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:20 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:20.172299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747bc14-7468-4518-a640-1029627cface admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:20.184479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747bc14-7468-4518-a640-1029627cface admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:20.244870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb965959-2ca2-45d1-8952-fedde169248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:20.245559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965959-2ca2-45d1-8952-fedde169248d admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.245846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965959-2ca2-45d1-8952-fedde169248d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.246120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb965959-2ca2-45d1-8952-fedde169248d admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.246819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb965959-2ca2-45d1-8952-fedde169248d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:20.248771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb965959-2ca2-45d1-8952-fedde169248d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:20.256128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:20.271694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1214/2430] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:20 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:20.276069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeb666-68d3-4c5a-819a-48a04cab06fa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:20.287443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeb666-68d3-4c5a-819a-48a04cab06fa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:20.292840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7297d91-ec9b-47de-b7be-7e8cfd045393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:20.293666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7297d91-ec9b-47de-b7be-7e8cfd045393 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.293934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7297d91-ec9b-47de-b7be-7e8cfd045393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.294252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7297d91-ec9b-47de-b7be-7e8cfd045393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.295150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7297d91-ec9b-47de-b7be-7e8cfd045393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:20.296851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7297d91-ec9b-47de-b7be-7e8cfd045393 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:20.308337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1217/2431] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:28:20.312848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb965959-2ca2-45d1-8952-fedde169248d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:20.323878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb965959-2ca2-45d1-8952-fedde169248d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:20.329360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc6676-4a1a-41f6-9f68-15da1239ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:20.330056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc6676-4a1a-41f6-9f68-15da1239ef43 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.330341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc6676-4a1a-41f6-9f68-15da1239ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.330621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc6676-4a1a-41f6-9f68-15da1239ef43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.331369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc6676-4a1a-41f6-9f68-15da1239ef43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:20.332986 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc6676-4a1a-41f6-9f68-15da1239ef43 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:20.349069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1215/2432] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:20.352942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7297d91-ec9b-47de-b7be-7e8cfd045393 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:20.358758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:20.365471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7297d91-ec9b-47de-b7be-7e8cfd045393 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:20.371570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:20.372483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.372763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.373039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] PATH_INFO: `/v3/projects/a0dcae29f5be488baacf188d90fa757d/users/917d3cbaedfe42e891997bf92b465224/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.390349 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0dcae29f5be488baacf188d90fa757d, user_id=917d3cbaedfe42e891997bf92b465224, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:20.393019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:20.410231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1218/2433] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:20 2022] PUT /identity/v3/projects/a0dcae29f5be488baacf188d90fa757d/users/917d3cbaedfe42e891997bf92b465224/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:20.421226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdc768-f3ee-422e-872d-91de47d35b6b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.421517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdc768-f3ee-422e-872d-91de47d35b6b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.421809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdc768-f3ee-422e-872d-91de47d35b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.453415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6bcdc768-f3ee-422e-872d-91de47d35b6b None None] MFA Rules not processed for user `917d3cbaedfe42e891997bf92b465224`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:20.455190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6bcdc768-f3ee-422e-872d-91de47d35b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:20.526229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1216/2434] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:20.538378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:20.540443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:20.623682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d702bc5-4073-443a-8742-16b0d321df68 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:20.629810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd13751-cdd8-4ad5-8a99-f754d52f08e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:20.630467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd13751-cdd8-4ad5-8a99-f754d52f08e1 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:20.630754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd13751-cdd8-4ad5-8a99-f754d52f08e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:20.631036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd13751-cdd8-4ad5-8a99-f754d52f08e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:20.639719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd13751-cdd8-4ad5-8a99-f754d52f08e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:20.641354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd13751-cdd8-4ad5-8a99-f754d52f08e1 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:20.651087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1219/2435] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:28:23.398162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdc768-f3ee-422e-872d-91de47d35b6b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:23.468646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdc768-f3ee-422e-872d-91de47d35b6b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:23.474043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:23.475496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:23.475774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:23.476047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] PATH_INFO: `/v3/users/917d3cbaedfe42e891997bf92b465224` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:23.477409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=917d3cbaedfe42e891997bf92b465224)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:23.479347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:23.529146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '917d3cbaedfe42e891997bf92b465224'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:23.543359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '917d3cbaedfe42e891997bf92b465224'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:23.544148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '917d3cbaedfe42e891997bf92b465224'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:23.550594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:23.555920 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '917d3cbaedfe42e891997bf92b465224'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:23.566321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '917d3cbaedfe42e891997bf92b465224'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:23.573459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1217/2436] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:23 2022] DELETE /identity/v3/users/917d3cbaedfe42e891997bf92b465224 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:28:23.578074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc54075-96d0-4400-a205-c83094a724a1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:23.578534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc54075-96d0-4400-a205-c83094a724a1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:23.578648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc54075-96d0-4400-a205-c83094a724a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:23.596063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9bc54075-96d0-4400-a205-c83094a724a1 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:23.597304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9bc54075-96d0-4400-a205-c83094a724a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:23.660602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1220/2437] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:28:23.671070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:23.673032 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:23.736286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c2d7e-00c0-45dd-834a-4e8369ffeb99 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:23.740651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d27bc-7efa-4ca9-b2b8-eb71b1ebc475 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:23.741342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d27bc-7efa-4ca9-b2b8-eb71b1ebc475 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:23.741617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d27bc-7efa-4ca9-b2b8-eb71b1ebc475 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:23.741896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d27bc-7efa-4ca9-b2b8-eb71b1ebc475 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:23.751429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d27bc-7efa-4ca9-b2b8-eb71b1ebc475 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:23.752854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d27bc-7efa-4ca9-b2b8-eb71b1ebc475 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:23.763145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1218/2438] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:24.071247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc54075-96d0-4400-a205-c83094a724a1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.072986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9bc54075-96d0-4400-a205-c83094a724a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:24.085829 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc54075-96d0-4400-a205-c83094a724a1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:24.099176 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:24.100714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.101069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.101382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] PATH_INFO: `/v3/projects/a0dcae29f5be488baacf188d90fa757d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.110765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0dcae29f5be488baacf188d90fa757d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:24.112791 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:24.166095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0dcae29f5be488baacf188d90fa757d', 'request_id': 'req-435232d0-8a6c-49e6-be3a-2be2e839582c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:24.171900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0dcae29f5be488baacf188d90fa757d', 'request_id': 'req-435232d0-8a6c-49e6-be3a-2be2e839582c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:24.173726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:24.182220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0dcae29f5be488baacf188d90fa757d', 'request_id': 'req-435232d0-8a6c-49e6-be3a-2be2e839582c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:24.196714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] The token cache is being invalidate because project a0dcae29f5be488baacf188d90fa757d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:24.197008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:24.197722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-435232d0-8a6c-49e6-be3a-2be2e839582c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:24.199300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1221/2439] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:24 2022] DELETE /identity/v3/projects/a0dcae29f5be488baacf188d90fa757d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:24.204402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829817-9246-49d7-a42a-5cacb4c37935 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.204686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829817-9246-49d7-a42a-5cacb4c37935 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.204962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829817-9246-49d7-a42a-5cacb4c37935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.225222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6a829817-9246-49d7-a42a-5cacb4c37935 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:24.226556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6a829817-9246-49d7-a42a-5cacb4c37935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:24.301624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1219/2440] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:24.311521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380115eb-6777-4fa4-9526-1f410d0efef0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.312034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380115eb-6777-4fa4-9526-1f410d0efef0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.312442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380115eb-6777-4fa4-9526-1f410d0efef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.334239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-380115eb-6777-4fa4-9526-1f410d0efef0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:24.335450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-380115eb-6777-4fa4-9526-1f410d0efef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:24.348001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1222/2441] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:24.355296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829817-9246-49d7-a42a-5cacb4c37935 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.370917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829817-9246-49d7-a42a-5cacb4c37935 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:24.376841 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:24.377547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.377831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.378106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.378843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:28:24.379154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:24.380987 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:24.395027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1220/2442] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:28:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:28:24.402500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380115eb-6777-4fa4-9526-1f410d0efef0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.418591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380115eb-6777-4fa4-9526-1f410d0efef0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:24.425870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf07cd-7a67-4d5c-b536-e14ec419681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:24.426776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf07cd-7a67-4d5c-b536-e14ec419681d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.427244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf07cd-7a67-4d5c-b536-e14ec419681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.427610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf07cd-7a67-4d5c-b536-e14ec419681d admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.428404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf07cd-7a67-4d5c-b536-e14ec419681d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:24.430470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf07cd-7a67-4d5c-b536-e14ec419681d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:24.459645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1223/2443] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:24 2022] POST /identity/v3/projects => generated 357 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:24.464170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.474804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f1583-7eea-43ec-8a15-636984edefce admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:24.543275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32f11c-c11d-45e5-b6aa-bf25818ce949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:24.544481 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32f11c-c11d-45e5-b6aa-bf25818ce949 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.544797 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32f11c-c11d-45e5-b6aa-bf25818ce949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.545063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef32f11c-c11d-45e5-b6aa-bf25818ce949 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.546131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32f11c-c11d-45e5-b6aa-bf25818ce949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:24.549090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef32f11c-c11d-45e5-b6aa-bf25818ce949 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:24.575266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1221/2444] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:24 2022] POST /identity/v3/users => generated 279 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:24.580778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf07cd-7a67-4d5c-b536-e14ec419681d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.595660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf07cd-7a67-4d5c-b536-e14ec419681d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:24.601148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389494-d6e1-4af6-aa18-bfdaebca9744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:24.601850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389494-d6e1-4af6-aa18-bfdaebca9744 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.602131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389494-d6e1-4af6-aa18-bfdaebca9744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.602403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389494-d6e1-4af6-aa18-bfdaebca9744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.603166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389494-d6e1-4af6-aa18-bfdaebca9744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:24.604758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1389494-d6e1-4af6-aa18-bfdaebca9744 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:24.617820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1224/2445] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:24.622191 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32f11c-c11d-45e5-b6aa-bf25818ce949 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.633493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef32f11c-c11d-45e5-b6aa-bf25818ce949 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:24.639346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd44d1-35d2-41d8-b93d-b708dc784ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:24.640031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd44d1-35d2-41d8-b93d-b708dc784ccc admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.640314 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd44d1-35d2-41d8-b93d-b708dc784ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.640593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd44d1-35d2-41d8-b93d-b708dc784ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.641368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd44d1-35d2-41d8-b93d-b708dc784ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:24.642970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd44d1-35d2-41d8-b93d-b708dc784ccc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:24.644350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:24.661331 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1222/2446] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:24.666081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389494-d6e1-4af6-aa18-bfdaebca9744 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.679563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1389494-d6e1-4af6-aa18-bfdaebca9744 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:24.684826 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0705a1-1f2f-4d6b-9e4d-6f32f7f9afbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:24.685770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0705a1-1f2f-4d6b-9e4d-6f32f7f9afbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.686058 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0705a1-1f2f-4d6b-9e4d-6f32f7f9afbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.686336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0705a1-1f2f-4d6b-9e4d-6f32f7f9afbf admin admin] PATH_INFO: `/v3/projects/1b9c47830cbe44c78a80bb13864600c6/users/a6614b3aad914b66a2f9d84e97fe14ab/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.702506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0705a1-1f2f-4d6b-9e4d-6f32f7f9afbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b9c47830cbe44c78a80bb13864600c6, user_id=a6614b3aad914b66a2f9d84e97fe14ab, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:24.705162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0705a1-1f2f-4d6b-9e4d-6f32f7f9afbf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:24.727131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1225/2447] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:24 2022] PUT /identity/v3/projects/1b9c47830cbe44c78a80bb13864600c6/users/a6614b3aad914b66a2f9d84e97fe14ab/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:24.738625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ad38-2d9b-47f4-b21c-2ab5cb52ca44 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.738915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ad38-2d9b-47f4-b21c-2ab5cb52ca44 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.739218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09ad38-2d9b-47f4-b21c-2ab5cb52ca44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.767660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-0d09ad38-2d9b-47f4-b21c-2ab5cb52ca44 None None] MFA Rules not processed for user `a6614b3aad914b66a2f9d84e97fe14ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:24.768927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-0d09ad38-2d9b-47f4-b21c-2ab5cb52ca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:24.841041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1223/2448] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:24 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:24.845720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0705a1-1f2f-4d6b-9e4d-6f32f7f9afbf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.919691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0705a1-1f2f-4d6b-9e4d-6f32f7f9afbf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:24.924904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b8e45-21ec-41f5-a42d-8f8660dec857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:24.925635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b8e45-21ec-41f5-a42d-8f8660dec857 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:24.925929 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b8e45-21ec-41f5-a42d-8f8660dec857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:24.926202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b8e45-21ec-41f5-a42d-8f8660dec857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:24.926916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b8e45-21ec-41f5-a42d-8f8660dec857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:24.928856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b8e45-21ec-41f5-a42d-8f8660dec857 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:24.958185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1226/2449] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:24 2022] POST /identity/v3/projects => generated 357 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:24.964361 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09ad38-2d9b-47f4-b21c-2ab5cb52ca44 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:24.979717 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09ad38-2d9b-47f4-b21c-2ab5cb52ca44 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:25.040859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cd03-2da1-4cb8-ae78-38aee259ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:25.041685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cd03-2da1-4cb8-ae78-38aee259ac6a admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:25.041969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cd03-2da1-4cb8-ae78-38aee259ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:25.042244 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323cd03-2da1-4cb8-ae78-38aee259ac6a admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:25.042949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323cd03-2da1-4cb8-ae78-38aee259ac6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:25.044874 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323cd03-2da1-4cb8-ae78-38aee259ac6a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:25.066251 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1224/2450] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:24 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:25.075480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b8e45-21ec-41f5-a42d-8f8660dec857 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:25.086430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b8e45-21ec-41f5-a42d-8f8660dec857 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:25.091280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe515cb-e6de-4093-b210-d99de8fbbaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:25.091962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe515cb-e6de-4093-b210-d99de8fbbaee admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:25.092240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe515cb-e6de-4093-b210-d99de8fbbaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:25.092523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe515cb-e6de-4093-b210-d99de8fbbaee admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:25.093282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe515cb-e6de-4093-b210-d99de8fbbaee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:25.094936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe515cb-e6de-4093-b210-d99de8fbbaee admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:25.105925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1227/2451] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:25.110733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cd03-2da1-4cb8-ae78-38aee259ac6a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:25.121734 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323cd03-2da1-4cb8-ae78-38aee259ac6a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:25.126829 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac36e4-1d5b-4a9f-9b6e-a64f44b7b59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:25.127718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac36e4-1d5b-4a9f-9b6e-a64f44b7b59f admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:25.127999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac36e4-1d5b-4a9f-9b6e-a64f44b7b59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:25.128272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac36e4-1d5b-4a9f-9b6e-a64f44b7b59f admin admin] PATH_INFO: `/v3/projects/9d55b8bbae674ba888ab696b939fe817/users/2bc40c94521c4584bb637b981e5f53cb/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:25.146772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac36e4-1d5b-4a9f-9b6e-a64f44b7b59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d55b8bbae674ba888ab696b939fe817, user_id=2bc40c94521c4584bb637b981e5f53cb, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:25.149418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac36e4-1d5b-4a9f-9b6e-a64f44b7b59f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:25.169715 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1225/2452] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:25 2022] PUT /identity/v3/projects/9d55b8bbae674ba888ab696b939fe817/users/2bc40c94521c4584bb637b981e5f53cb/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:28:25.175443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe515cb-e6de-4093-b210-d99de8fbbaee admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:25.186996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe515cb-e6de-4093-b210-d99de8fbbaee admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:25.244346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:25.255155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf7bb6-53b8-44af-9ba0-43dfa9b0f8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:25.256775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf7bb6-53b8-44af-9ba0-43dfa9b0f8af admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:25.256775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf7bb6-53b8-44af-9ba0-43dfa9b0f8af admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:25.257010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf7bb6-53b8-44af-9ba0-43dfa9b0f8af admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:25.258073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf7bb6-53b8-44af-9ba0-43dfa9b0f8af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:25.261217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf7bb6-53b8-44af-9ba0-43dfa9b0f8af admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:25.275637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1228/2453] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:25 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:28:25.279975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac36e4-1d5b-4a9f-9b6e-a64f44b7b59f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:25.290863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac36e4-1d5b-4a9f-9b6e-a64f44b7b59f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:25.296187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:25.297062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:25.297371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:25.297634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] PATH_INFO: `/v3/projects/9d55b8bbae674ba888ab696b939fe817/users/2bc40c94521c4584bb637b981e5f53cb/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:25.299652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d55b8bbae674ba888ab696b939fe817, user_id=2bc40c94521c4584bb637b981e5f53cb, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:25.302300 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:25.314402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1226/2454] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:25 2022] PUT /identity/v3/projects/9d55b8bbae674ba888ab696b939fe817/users/2bc40c94521c4584bb637b981e5f53cb/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:25.324733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77dc3-b4c6-42af-a90e-3fa2084dffb3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:25.325014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77dc3-b4c6-42af-a90e-3fa2084dffb3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:25.325346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b77dc3-b4c6-42af-a90e-3fa2084dffb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:25.355127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-71b77dc3-b4c6-42af-a90e-3fa2084dffb3 None None] MFA Rules not processed for user `2bc40c94521c4584bb637b981e5f53cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:25.356347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-71b77dc3-b4c6-42af-a90e-3fa2084dffb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:25.434932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1229/2455] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:25 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:25.447376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:25.450019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:25.506932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:25.526527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa7264-3f65-4194-b196-01e05bdfafdd admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:25.530914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e023393-1f79-491c-a6f5-ca0cce666c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:25.531636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e023393-1f79-491c-a6f5-ca0cce666c10 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:25.531980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e023393-1f79-491c-a6f5-ca0cce666c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:25.532133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e023393-1f79-491c-a6f5-ca0cce666c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:25.542759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e023393-1f79-491c-a6f5-ca0cce666c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:25.544172 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e023393-1f79-491c-a6f5-ca0cce666c10 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:25.554372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1227/2456] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:25 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:25.637927 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b77dc3-b4c6-42af-a90e-3fa2084dffb3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:25.653564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b77dc3-b4c6-42af-a90e-3fa2084dffb3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:25.659889 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:25.660836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:25.661219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:25.661618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:25.721427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:25.722861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:25.732192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1230/2457] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:25 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:35.549645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=9 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:35.728049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=65 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:39.851412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2ccd-c8ab-4c0d-a0c3-202e2c18a6e4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:39.851936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2ccd-c8ab-4c0d-a0c3-202e2c18a6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:39.851936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a2ccd-c8ab-4c0d-a0c3-202e2c18a6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:39.872793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-128a2ccd-c8ab-4c0d-a0c3-202e2c18a6e4 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:39.874192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-128a2ccd-c8ab-4c0d-a0c3-202e2c18a6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:39.944531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1228/2458] 10.222.0.66 () {58 vars in 967 bytes} [Thu Jun 16 17:28:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:40.000201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:40.001940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:40.114287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5707469-33c2-4c81-982c-2b0385f8a16b service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:40.119609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb2032-7422-4b68-bb8f-177434337a9a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:40.120333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb2032-7422-4b68-bb8f-177434337a9a service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:40.120617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb2032-7422-4b68-bb8f-177434337a9a service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:40.120901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb2032-7422-4b68-bb8f-177434337a9a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:40.131915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb2032-7422-4b68-bb8f-177434337a9a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:40.134311 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb2032-7422-4b68-bb8f-177434337a9a service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:40.144011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1231/2459] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:28:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:28:41.153910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a2ccd-c8ab-4c0d-a0c3-202e2c18a6e4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:41.232515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a2ccd-c8ab-4c0d-a0c3-202e2c18a6e4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:41.250060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:41.251033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:41.251318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:41.251596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] PATH_INFO: `/v3/users/a6614b3aad914b66a2f9d84e97fe14ab` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:41.252917 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6614b3aad914b66a2f9d84e97fe14ab)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:41.255221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:41.324217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6614b3aad914b66a2f9d84e97fe14ab'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:41.338527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6614b3aad914b66a2f9d84e97fe14ab'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:41.339617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6614b3aad914b66a2f9d84e97fe14ab'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:41.351724 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6614b3aad914b66a2f9d84e97fe14ab'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:41.362141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6614b3aad914b66a2f9d84e97fe14ab'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:41.371119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1229/2460] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:41 2022] DELETE /identity/v3/users/a6614b3aad914b66a2f9d84e97fe14ab => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:41.376166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58befc3-c2e6-4591-a203-193b0cb850c0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:41.376925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58befc3-c2e6-4591-a203-193b0cb850c0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:41.376925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58befc3-c2e6-4591-a203-193b0cb850c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:41.398081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f58befc3-c2e6-4591-a203-193b0cb850c0 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:41.399213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f58befc3-c2e6-4591-a203-193b0cb850c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:41.465954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1232/2461] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:41.479153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:41.480744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:41.520913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:41.566228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91977501-8047-4b3e-a38e-f33e3cef6545 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:41.571100 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327447de-6b34-4ef2-b333-fce98b1909f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:41.571781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327447de-6b34-4ef2-b333-fce98b1909f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:41.572063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327447de-6b34-4ef2-b333-fce98b1909f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:41.572340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327447de-6b34-4ef2-b333-fce98b1909f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:41.585413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327447de-6b34-4ef2-b333-fce98b1909f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:41.586963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327447de-6b34-4ef2-b333-fce98b1909f6 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:41.597962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1230/2462] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:41.908662 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58befc3-c2e6-4591-a203-193b0cb850c0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:41.910320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f58befc3-c2e6-4591-a203-193b0cb850c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:41.923484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58befc3-c2e6-4591-a203-193b0cb850c0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:41.929930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:41.931177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:41.931672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:41.940190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] PATH_INFO: `/v3/projects/1b9c47830cbe44c78a80bb13864600c6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:41.941808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b9c47830cbe44c78a80bb13864600c6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:41.944111 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:41.999645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b9c47830cbe44c78a80bb13864600c6', 'request_id': 'req-6adc476f-2cfb-4324-b530-28aec7e22d67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.005520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b9c47830cbe44c78a80bb13864600c6', 'request_id': 'req-6adc476f-2cfb-4324-b530-28aec7e22d67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.015217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b9c47830cbe44c78a80bb13864600c6', 'request_id': 'req-6adc476f-2cfb-4324-b530-28aec7e22d67'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.027683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] The token cache is being invalidate because project 1b9c47830cbe44c78a80bb13864600c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:42.027971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.028618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.030400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1233/2463] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:41 2022] DELETE /identity/v3/projects/1b9c47830cbe44c78a80bb13864600c6 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:42.034212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327447de-6b34-4ef2-b333-fce98b1909f6 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:42.045009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327447de-6b34-4ef2-b333-fce98b1909f6 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:42.107318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:42.108230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.108497 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.108824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] PATH_INFO: `/v3/users/2bc40c94521c4584bb637b981e5f53cb` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.110472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bc40c94521c4584bb637b981e5f53cb)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:42.112974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:42.167974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bc40c94521c4584bb637b981e5f53cb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.182448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bc40c94521c4584bb637b981e5f53cb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.183357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bc40c94521c4584bb637b981e5f53cb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.191808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bc40c94521c4584bb637b981e5f53cb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.203696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-cbce8eaf-767f-4e63-814f-22ff31374326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bc40c94521c4584bb637b981e5f53cb'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.210912 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1231/2464] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:42 2022] DELETE /identity/v3/users/2bc40c94521c4584bb637b981e5f53cb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:42.525929 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:42.528867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:42.552746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:42.559923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc476f-2cfb-4324-b530-28aec7e22d67 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:42.640187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:42.641429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.642833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.643135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] PATH_INFO: `/v3/projects/9d55b8bbae674ba888ab696b939fe817` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.644467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d55b8bbae674ba888ab696b939fe817)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:42.646592 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:42.707057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d55b8bbae674ba888ab696b939fe817', 'request_id': 'req-5bd18833-4d7f-4f56-a0d1-74610af61e03'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.713773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d55b8bbae674ba888ab696b939fe817', 'request_id': 'req-5bd18833-4d7f-4f56-a0d1-74610af61e03'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.725475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d55b8bbae674ba888ab696b939fe817', 'request_id': 'req-5bd18833-4d7f-4f56-a0d1-74610af61e03'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.739318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] The token cache is being invalidate because project 9d55b8bbae674ba888ab696b939fe817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:42.739675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.740960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5bd18833-4d7f-4f56-a0d1-74610af61e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:42.742502 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1234/2465] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:42 2022] DELETE /identity/v3/projects/9d55b8bbae674ba888ab696b939fe817 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:42.747676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913dfd-9c66-4c2c-9d08-27ce10d64b19 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.748055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913dfd-9c66-4c2c-9d08-27ce10d64b19 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.748254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913dfd-9c66-4c2c-9d08-27ce10d64b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.768165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f7913dfd-9c66-4c2c-9d08-27ce10d64b19 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:42.769855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f7913dfd-9c66-4c2c-9d08-27ce10d64b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:42.829476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:42.866478 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1232/2466] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:42.874737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838f239-9b39-48af-91dd-5064d63f0474 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.875032 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838f239-9b39-48af-91dd-5064d63f0474 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.875296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b838f239-9b39-48af-91dd-5064d63f0474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.895478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b838f239-9b39-48af-91dd-5064d63f0474 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:42.897107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b838f239-9b39-48af-91dd-5064d63f0474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:42.906487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1235/2467] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:28:42.910697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7913dfd-9c66-4c2c-9d08-27ce10d64b19 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:42.922631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7913dfd-9c66-4c2c-9d08-27ce10d64b19 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:42.929316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:42.930216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.930503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.930790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.931557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:28:42.931870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:42.933553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:42.948064 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1233/2468] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:28:42.953206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838f239-9b39-48af-91dd-5064d63f0474 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:42.964704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b838f239-9b39-48af-91dd-5064d63f0474 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:42.971925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c74ef-3b9c-4757-be67-61999e9fcc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:42.972811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c74ef-3b9c-4757-be67-61999e9fcc44 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:42.973206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c74ef-3b9c-4757-be67-61999e9fcc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:42.973733 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c74ef-3b9c-4757-be67-61999e9fcc44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:42.974271 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c74ef-3b9c-4757-be67-61999e9fcc44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:42.977303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c74ef-3b9c-4757-be67-61999e9fcc44 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.020905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1236/2469] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:42 2022] POST /identity/v3/projects => generated 369 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:43.025959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.037083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb93273-e256-4d9d-a20b-9068a15cb129 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.105055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22eff0-a3ea-4122-a879-447cd6feb506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.106087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22eff0-a3ea-4122-a879-447cd6feb506 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.106392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22eff0-a3ea-4122-a879-447cd6feb506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.106664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22eff0-a3ea-4122-a879-447cd6feb506 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.107372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22eff0-a3ea-4122-a879-447cd6feb506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.109365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22eff0-a3ea-4122-a879-447cd6feb506 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.137568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1234/2470] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:43 2022] POST /identity/v3/users => generated 285 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:43.141914 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c74ef-3b9c-4757-be67-61999e9fcc44 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.153696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c74ef-3b9c-4757-be67-61999e9fcc44 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.159956 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f75365-4383-4083-914f-fc1468906295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.161721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f75365-4383-4083-914f-fc1468906295 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.161721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f75365-4383-4083-914f-fc1468906295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.161721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f75365-4383-4083-914f-fc1468906295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.162775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f75365-4383-4083-914f-fc1468906295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.164519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f75365-4383-4083-914f-fc1468906295 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.187169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1237/2471] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:43.191016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22eff0-a3ea-4122-a879-447cd6feb506 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.203018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22eff0-a3ea-4122-a879-447cd6feb506 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.227421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1e1d5-854c-4024-bb67-7c2bca2cf374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.228162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1e1d5-854c-4024-bb67-7c2bca2cf374 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.228671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1e1d5-854c-4024-bb67-7c2bca2cf374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.229019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1e1d5-854c-4024-bb67-7c2bca2cf374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.229884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1e1d5-854c-4024-bb67-7c2bca2cf374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.231628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1e1d5-854c-4024-bb67-7c2bca2cf374 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.268212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1235/2472] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:43.269593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f75365-4383-4083-914f-fc1468906295 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.284768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f75365-4383-4083-914f-fc1468906295 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.291560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa11f-189e-4a1e-bd74-d351ec69704a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.293109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa11f-189e-4a1e-bd74-d351ec69704a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.293109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa11f-189e-4a1e-bd74-d351ec69704a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.293109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054fa11f-189e-4a1e-bd74-d351ec69704a admin admin] PATH_INFO: `/v3/projects/af54aa80919847faac2a98437afcf99e/users/a20cf122f8f3436c8ecf28860483e59f/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.318407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa11f-189e-4a1e-bd74-d351ec69704a admin admin] RBAC: Authorizing `identity:create_grant(project_id=af54aa80919847faac2a98437afcf99e, user_id=a20cf122f8f3436c8ecf28860483e59f, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.321720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054fa11f-189e-4a1e-bd74-d351ec69704a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.351322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1238/2473] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:43 2022] PUT /identity/v3/projects/af54aa80919847faac2a98437afcf99e/users/a20cf122f8f3436c8ecf28860483e59f/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:43.355044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d717f-a45c-40b5-ae61-bbd281372383 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.355354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d717f-a45c-40b5-ae61-bbd281372383 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.355660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d717f-a45c-40b5-ae61-bbd281372383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.384980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-579d717f-a45c-40b5-ae61-bbd281372383 None None] MFA Rules not processed for user `a20cf122f8f3436c8ecf28860483e59f`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:43.386242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-579d717f-a45c-40b5-ae61-bbd281372383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:43.466416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1236/2474] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:43.471024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa11f-189e-4a1e-bd74-d351ec69704a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.512765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:43.563652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054fa11f-189e-4a1e-bd74-d351ec69704a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.570104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9983f-a911-4020-91ee-c176c8031a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.571035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9983f-a911-4020-91ee-c176c8031a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.571324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9983f-a911-4020-91ee-c176c8031a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.571633 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9983f-a911-4020-91ee-c176c8031a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.572357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9983f-a911-4020-91ee-c176c8031a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.574360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9983f-a911-4020-91ee-c176c8031a09 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.611294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1239/2475] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:43 2022] POST /identity/v3/projects => generated 369 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:43.612394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d717f-a45c-40b5-ae61-bbd281372383 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.628973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d717f-a45c-40b5-ae61-bbd281372383 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.697539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:43.719656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc9411-bee0-4f2b-9460-720f28ae2ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.720432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9411-bee0-4f2b-9460-720f28ae2ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.720819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9411-bee0-4f2b-9460-720f28ae2ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.721194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc9411-bee0-4f2b-9460-720f28ae2ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.722053 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc9411-bee0-4f2b-9460-720f28ae2ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.724320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc9411-bee0-4f2b-9460-720f28ae2ca7 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.754656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1237/2476] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:43 2022] POST /identity/v3/users => generated 291 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:28:43.758867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9983f-a911-4020-91ee-c176c8031a09 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.775950 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9983f-a911-4020-91ee-c176c8031a09 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.782400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de15a9-f247-4261-8c6a-86e8252cf263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.783102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de15a9-f247-4261-8c6a-86e8252cf263 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.783384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de15a9-f247-4261-8c6a-86e8252cf263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.783708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de15a9-f247-4261-8c6a-86e8252cf263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.784541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de15a9-f247-4261-8c6a-86e8252cf263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.786436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de15a9-f247-4261-8c6a-86e8252cf263 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.799396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1240/2477] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:43.804025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc9411-bee0-4f2b-9460-720f28ae2ca7 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.820073 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc9411-bee0-4f2b-9460-720f28ae2ca7 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.826458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27862-732f-4315-8a67-f9fa6e6b94fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.828038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27862-732f-4315-8a67-f9fa6e6b94fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.828497 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27862-732f-4315-8a67-f9fa6e6b94fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.828782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27862-732f-4315-8a67-f9fa6e6b94fa admin admin] PATH_INFO: `/v3/projects/1d48e603f2854700ad33a597d8794bec/users/ab31ee9ec2f842fda33f243b96a2084b/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.857607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27862-732f-4315-8a67-f9fa6e6b94fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d48e603f2854700ad33a597d8794bec, user_id=ab31ee9ec2f842fda33f243b96a2084b, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.861568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27862-732f-4315-8a67-f9fa6e6b94fa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:43.886450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1238/2478] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:43 2022] PUT /identity/v3/projects/1d48e603f2854700ad33a597d8794bec/users/ab31ee9ec2f842fda33f243b96a2084b/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:43.888871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de15a9-f247-4261-8c6a-86e8252cf263 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:43.907189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de15a9-f247-4261-8c6a-86e8252cf263 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:43.973973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ab044-1200-4e8d-8adf-36df654fd154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:43.974776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ab044-1200-4e8d-8adf-36df654fd154 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:43.975332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ab044-1200-4e8d-8adf-36df654fd154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:43.975628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ab044-1200-4e8d-8adf-36df654fd154 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:43.977143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ab044-1200-4e8d-8adf-36df654fd154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:43.979404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ab044-1200-4e8d-8adf-36df654fd154 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:44.000629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27862-732f-4315-8a67-f9fa6e6b94fa admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:44.005261 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1241/2479] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:44.014795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27862-732f-4315-8a67-f9fa6e6b94fa admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:44.020049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:44.020604 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:44.020884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:44.021563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] PATH_INFO: `/v3/projects/1d48e603f2854700ad33a597d8794bec/users/ab31ee9ec2f842fda33f243b96a2084b/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:44.023866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d48e603f2854700ad33a597d8794bec, user_id=ab31ee9ec2f842fda33f243b96a2084b, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:44.026651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:44.040257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1239/2480] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:43 2022] PUT /identity/v3/projects/1d48e603f2854700ad33a597d8794bec/users/ab31ee9ec2f842fda33f243b96a2084b/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:44.052113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2e70d-4aeb-491c-af7f-e0918c55eb09 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:44.052480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2e70d-4aeb-491c-af7f-e0918c55eb09 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:44.052759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2e70d-4aeb-491c-af7f-e0918c55eb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:44.084542 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a5f2e70d-4aeb-491c-af7f-e0918c55eb09 None None] MFA Rules not processed for user `ab31ee9ec2f842fda33f243b96a2084b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:44.086053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a5f2e70d-4aeb-491c-af7f-e0918c55eb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:44.185078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1242/2481] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:44.195189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:44.197090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:44.269259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09213123-7663-4b02-baec-a3ca3188e388 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:44.274167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d9a20-7bf0-4395-8b64-6978cd3b4c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:44.274879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9a20-7bf0-4395-8b64-6978cd3b4c5e service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:44.275163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9a20-7bf0-4395-8b64-6978cd3b4c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:44.275447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d9a20-7bf0-4395-8b64-6978cd3b4c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:44.287669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d9a20-7bf0-4395-8b64-6978cd3b4c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:44.289540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d9a20-7bf0-4395-8b64-6978cd3b4c5e service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:44.300615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1240/2482] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:45.890937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2e70d-4aeb-491c-af7f-e0918c55eb09 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:45.915663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:45.971810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2e70d-4aeb-491c-af7f-e0918c55eb09 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:45.978194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:45.979157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:45.979443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:45.979720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] PATH_INFO: `/v3/users/a20cf122f8f3436c8ecf28860483e59f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:45.980981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20cf122f8f3436c8ecf28860483e59f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:45.983034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:46.038258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20cf122f8f3436c8ecf28860483e59f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.038960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20cf122f8f3436c8ecf28860483e59f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.039567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20cf122f8f3436c8ecf28860483e59f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.053869 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20cf122f8f3436c8ecf28860483e59f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.062721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20cf122f8f3436c8ecf28860483e59f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.078717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1243/2483] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:45 2022] DELETE /identity/v3/users/a20cf122f8f3436c8ecf28860483e59f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:46.083263 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faadd-c6b4-4507-9331-b5eb622f35ce None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:46.084260 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faadd-c6b4-4507-9331-b5eb622f35ce None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:46.084714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60faadd-c6b4-4507-9331-b5eb622f35ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:46.106675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e60faadd-c6b4-4507-9331-b5eb622f35ce None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:46.107959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e60faadd-c6b4-4507-9331-b5eb622f35ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:46.178446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1241/2484] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:46.192031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:46.194665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:46.268078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de37c56e-736f-4e4a-82e8-03fb9e42b774 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:46.272549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368581d-ad54-4a64-806b-932f594afc3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:46.273431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368581d-ad54-4a64-806b-932f594afc3a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:46.273721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368581d-ad54-4a64-806b-932f594afc3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:46.274017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368581d-ad54-4a64-806b-932f594afc3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:46.284936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368581d-ad54-4a64-806b-932f594afc3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:46.286430 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9368581d-ad54-4a64-806b-932f594afc3a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:46.301233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1244/2485] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:46.629411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60faadd-c6b4-4507-9331-b5eb622f35ce None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:46.631348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e60faadd-c6b4-4507-9331-b5eb622f35ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:46.648174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60faadd-c6b4-4507-9331-b5eb622f35ce None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:46.665037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:46.666483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:46.667186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:46.667562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] PATH_INFO: `/v3/projects/af54aa80919847faac2a98437afcf99e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:46.669618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af54aa80919847faac2a98437afcf99e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:46.673793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:46.730222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:46.753318 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af54aa80919847faac2a98437afcf99e', 'request_id': 'req-4d86200b-713a-40f5-8934-fe81a00e6358'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.766601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af54aa80919847faac2a98437afcf99e', 'request_id': 'req-4d86200b-713a-40f5-8934-fe81a00e6358'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.778030 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af54aa80919847faac2a98437afcf99e', 'request_id': 'req-4d86200b-713a-40f5-8934-fe81a00e6358'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.825767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] The token cache is being invalidate because project af54aa80919847faac2a98437afcf99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:46.826135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.826904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:46.828767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1242/2486] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:46 2022] DELETE /identity/v3/projects/af54aa80919847faac2a98437afcf99e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:46.833344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368581d-ad54-4a64-806b-932f594afc3a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:46.850730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368581d-ad54-4a64-806b-932f594afc3a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:46.890043 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:46.939255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:46.940755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:46.941438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:46.941978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] PATH_INFO: `/v3/users/ab31ee9ec2f842fda33f243b96a2084b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:46.944860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab31ee9ec2f842fda33f243b96a2084b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:46.950344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:47.064247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab31ee9ec2f842fda33f243b96a2084b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.065482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab31ee9ec2f842fda33f243b96a2084b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.066837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab31ee9ec2f842fda33f243b96a2084b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.090995 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab31ee9ec2f842fda33f243b96a2084b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.106105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f5c39e-3c83-4923-94bb-bf0160020f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab31ee9ec2f842fda33f243b96a2084b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.132023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1245/2487] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:46 2022] DELETE /identity/v3/users/ab31ee9ec2f842fda33f243b96a2084b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:28:47.520788 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:47.535160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:47.555723 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86200b-713a-40f5-8934-fe81a00e6358 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:47.692114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:47.694065 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:47.697215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:47.698134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] PATH_INFO: `/v3/projects/1d48e603f2854700ad33a597d8794bec` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:47.699915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d48e603f2854700ad33a597d8794bec)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:47.703682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:47.811182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d48e603f2854700ad33a597d8794bec', 'request_id': 'req-5c13e0f4-8461-4db3-b36b-978a746dda2f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.822397 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d48e603f2854700ad33a597d8794bec', 'request_id': 'req-5c13e0f4-8461-4db3-b36b-978a746dda2f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.830930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d48e603f2854700ad33a597d8794bec', 'request_id': 'req-5c13e0f4-8461-4db3-b36b-978a746dda2f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.848304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] The token cache is being invalidate because project 1d48e603f2854700ad33a597d8794bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:47.848909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.851825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5c13e0f4-8461-4db3-b36b-978a746dda2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:47.855194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1243/2488] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:47 2022] DELETE /identity/v3/projects/1d48e603f2854700ad33a597d8794bec => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:47.860443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e12dc-b664-428a-a23b-a8e215a7cfcb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:47.861039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e12dc-b664-428a-a23b-a8e215a7cfcb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:47.861365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e12dc-b664-428a-a23b-a8e215a7cfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:47.881608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-141e12dc-b664-428a-a23b-a8e215a7cfcb None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:47.883108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-141e12dc-b664-428a-a23b-a8e215a7cfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:47.982746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1246/2489] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:47.991443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266be3-5f50-4521-abf5-2ca6fe72c14f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:47.991652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266be3-5f50-4521-abf5-2ca6fe72c14f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:47.991925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266be3-5f50-4521-abf5-2ca6fe72c14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.024183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-35266be3-5f50-4521-abf5-2ca6fe72c14f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:48.026161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-35266be3-5f50-4521-abf5-2ca6fe72c14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:48.041265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1244/2490] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:28:48.042558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e12dc-b664-428a-a23b-a8e215a7cfcb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.055828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e12dc-b664-428a-a23b-a8e215a7cfcb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.062576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.063864 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.064434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.064907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.065938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:28:48.066367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.068816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.091591 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1247/2491] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:28:48.091973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266be3-5f50-4521-abf5-2ca6fe72c14f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.103017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266be3-5f50-4521-abf5-2ca6fe72c14f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.108878 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.109797 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.110229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.110672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.111831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.114398 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.170664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1245/2492] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:48 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:28:48.175080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.186794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d0357-ca4d-48e9-b20d-d210cd16d798 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.241845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.246098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:48.250213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:48.286147 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7834-baf1-411f-b966-00704b18af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.286895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7834-baf1-411f-b966-00704b18af97 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.287256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7834-baf1-411f-b966-00704b18af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.287597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d7834-baf1-411f-b966-00704b18af97 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.288373 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d7834-baf1-411f-b966-00704b18af97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.290421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d7834-baf1-411f-b966-00704b18af97 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.305047 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:48.326809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1248/2493] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:48 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:48.334847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7834-baf1-411f-b966-00704b18af97 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.346439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d7834-baf1-411f-b966-00704b18af97 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.351734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6dac0-a232-4596-adf8-442df95bee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.352641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6dac0-a232-4596-adf8-442df95bee98 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.352855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6dac0-a232-4596-adf8-442df95bee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.353287 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de6dac0-a232-4596-adf8-442df95bee98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.354069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6dac0-a232-4596-adf8-442df95bee98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.355745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de6dac0-a232-4596-adf8-442df95bee98 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.357900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d632b9-d1d8-4492-a117-688874d12c1a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.364373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.365350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.365635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.365914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] PATH_INFO: `/v3/users/71df16d1e792468585702fd688b8d3f2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.367233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71df16d1e792468585702fd688b8d3f2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.369218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1249/2494] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:28:48.369370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.372981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6dac0-a232-4596-adf8-442df95bee98 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.383977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de6dac0-a232-4596-adf8-442df95bee98 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.389019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1e88a-6b83-4ed3-82c3-8310ea25910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.389815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e88a-6b83-4ed3-82c3-8310ea25910a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.390103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e88a-6b83-4ed3-82c3-8310ea25910a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.390524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f1e88a-6b83-4ed3-82c3-8310ea25910a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.391282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1e88a-6b83-4ed3-82c3-8310ea25910a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.393002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f1e88a-6b83-4ed3-82c3-8310ea25910a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.404129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1250/2495] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:28:48.408638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1e88a-6b83-4ed3-82c3-8310ea25910a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.419476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f1e88a-6b83-4ed3-82c3-8310ea25910a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.426609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1f638-2480-47ce-82da-d64b1332b574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.427093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71df16d1e792468585702fd688b8d3f2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.427722 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1f638-2480-47ce-82da-d64b1332b574 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.428069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1f638-2480-47ce-82da-d64b1332b574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.428519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1f638-2480-47ce-82da-d64b1332b574 admin admin] PATH_INFO: `/v3/projects/f8ae0c1b586b4cca8267f0e1c872d331/users/65f5a71ff2ef4ec2a7ca6df1717e3b9a/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.449012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71df16d1e792468585702fd688b8d3f2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.450246 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1f638-2480-47ce-82da-d64b1332b574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ae0c1b586b4cca8267f0e1c872d331, user_id=65f5a71ff2ef4ec2a7ca6df1717e3b9a, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.452922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1f638-2480-47ce-82da-d64b1332b574 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.453675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71df16d1e792468585702fd688b8d3f2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.466286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71df16d1e792468585702fd688b8d3f2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.473005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1251/2496] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:48 2022] PUT /identity/v3/projects/f8ae0c1b586b4cca8267f0e1c872d331/users/65f5a71ff2ef4ec2a7ca6df1717e3b9a/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:28:48.484793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71df16d1e792468585702fd688b8d3f2'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.488100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eb0e7-142e-432a-b2a8-49bb961c3745 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.488984 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eb0e7-142e-432a-b2a8-49bb961c3745 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.489438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103eb0e7-142e-432a-b2a8-49bb961c3745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.497572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1246/2497] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:48 2022] DELETE /identity/v3/users/71df16d1e792468585702fd688b8d3f2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:28:48.522329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-103eb0e7-142e-432a-b2a8-49bb961c3745 None None] MFA Rules not processed for user `65f5a71ff2ef4ec2a7ca6df1717e3b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:48.523856 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-103eb0e7-142e-432a-b2a8-49bb961c3745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:48.604227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1252/2498] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:28:48.605894 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.607892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:48.690866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377692c-18b8-4f08-bb50-33c2643b7e16 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.697387 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eb0e7-142e-432a-b2a8-49bb961c3745 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.699442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-103eb0e7-142e-432a-b2a8-49bb961c3745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:48.701661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea844a-e660-485e-abc3-0e4bba254b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.702415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea844a-e660-485e-abc3-0e4bba254b3d admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.702705 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea844a-e660-485e-abc3-0e4bba254b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.702990 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea844a-e660-485e-abc3-0e4bba254b3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.703706 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea844a-e660-485e-abc3-0e4bba254b3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.705642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea844a-e660-485e-abc3-0e4bba254b3d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.724938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103eb0e7-142e-432a-b2a8-49bb961c3745 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.737837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.738751 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.739040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.739312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] PATH_INFO: `/v3/projects/546929b1810a4d20b273f3e5492ec6ed` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.745089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=546929b1810a4d20b273f3e5492ec6ed)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.745089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.770020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1247/2499] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:48 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:48.774193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea844a-e660-485e-abc3-0e4bba254b3d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.812183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:48.830177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546929b1810a4d20b273f3e5492ec6ed', 'request_id': 'req-6498d783-6fab-4c4a-8e39-0f313e362ec6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.839142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546929b1810a4d20b273f3e5492ec6ed', 'request_id': 'req-6498d783-6fab-4c4a-8e39-0f313e362ec6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.848331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546929b1810a4d20b273f3e5492ec6ed', 'request_id': 'req-6498d783-6fab-4c4a-8e39-0f313e362ec6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.858096 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea844a-e660-485e-abc3-0e4bba254b3d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:48.861136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:48.865923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] The token cache is being invalidate because project 546929b1810a4d20b273f3e5492ec6ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:48.866613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.867793 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:48.870840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1253/2500] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:48 2022] DELETE /identity/v3/projects/546929b1810a4d20b273f3e5492ec6ed => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:48.878392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.880195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:48.950644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:48.951692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:48.952179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:48.952629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:48.953827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:48.957157 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:48.990266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1248/2501] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:48 2022] POST /identity/v3/users => generated 296 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:48.994951 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:48.997674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:48.998367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498d783-6fab-4c4a-8e39-0f313e362ec6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:49.004978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:49.006644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.007244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.007841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] PATH_INFO: `/v3/users/5071d8cc509549dd98077dbf59591d35` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.009134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5071d8cc509549dd98077dbf59591d35)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:49.011358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:49.014565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a6c10-d491-4bf4-aee6-f33cfee2fbfa admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:49.020159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a598a4-a025-4eb4-9b95-0a065adce28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:49.020879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a598a4-a025-4eb4-9b95-0a065adce28b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.021302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a598a4-a025-4eb4-9b95-0a065adce28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.021612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a598a4-a025-4eb4-9b95-0a065adce28b admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.022309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a598a4-a025-4eb4-9b95-0a065adce28b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:49.023991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a598a4-a025-4eb4-9b95-0a065adce28b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:49.038537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1249/2502] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:49.043612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a598a4-a025-4eb4-9b95-0a065adce28b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:49.058576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a598a4-a025-4eb4-9b95-0a065adce28b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:49.070891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:49.070891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.070891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.070891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] PATH_INFO: `/v3/projects/b80a8c3c496d4b87b042708645c1088b/users/4a32b040b48746aa8bf040e3c98643ef/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.092487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80a8c3c496d4b87b042708645c1088b, user_id=4a32b040b48746aa8bf040e3c98643ef, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:49.098900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:49.101597 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5071d8cc509549dd98077dbf59591d35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.102578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5071d8cc509549dd98077dbf59591d35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.103269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5071d8cc509549dd98077dbf59591d35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.120435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1250/2503] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:49 2022] PUT /identity/v3/projects/b80a8c3c496d4b87b042708645c1088b/users/4a32b040b48746aa8bf040e3c98643ef/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:49.121850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5071d8cc509549dd98077dbf59591d35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.124210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:49.126365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:49.134672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5071d8cc509549dd98077dbf59591d35'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.156895 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1254/2504] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:48 2022] DELETE /identity/v3/users/5071d8cc509549dd98077dbf59591d35 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:49.226351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bfa9b-c3ff-419c-86b6-360caa87d81c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:49.296532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:49.331605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:49.332866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.333154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.333656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.334587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:49.336270 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:49.352393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1251/2505] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:28:49.356726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:49.370923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80368959-40da-405c-9549-f4d651f6761a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:49.375721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96596580-200f-4594-b2d5-c7f2d0ba3d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:49.377219 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596580-200f-4594-b2d5-c7f2d0ba3d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.377830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596580-200f-4594-b2d5-c7f2d0ba3d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.377830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96596580-200f-4594-b2d5-c7f2d0ba3d4d admin admin] PATH_INFO: `/v3/projects/b80a8c3c496d4b87b042708645c1088b/users/4a32b040b48746aa8bf040e3c98643ef/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.380399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96596580-200f-4594-b2d5-c7f2d0ba3d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80a8c3c496d4b87b042708645c1088b, user_id=4a32b040b48746aa8bf040e3c98643ef, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:49.383461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96596580-200f-4594-b2d5-c7f2d0ba3d4d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:49.403479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1255/2506] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:49 2022] PUT /identity/v3/projects/b80a8c3c496d4b87b042708645c1088b/users/4a32b040b48746aa8bf040e3c98643ef/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:28:49.414804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:49.417003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc2e5e-c41e-48ea-b483-d835ed9c0fd6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.417800 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc2e5e-c41e-48ea-b483-d835ed9c0fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.418024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc2e5e-c41e-48ea-b483-d835ed9c0fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.422041 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:49.436162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32104abf-c1c3-4a4c-a1e8-d3a59c8e948e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:49.466082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5dfc2e5e-c41e-48ea-b483-d835ed9c0fd6 None None] MFA Rules not processed for user `4a32b040b48746aa8bf040e3c98643ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:49.468327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5dfc2e5e-c41e-48ea-b483-d835ed9c0fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:49.504749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:49.505881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.506219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.506510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] PATH_INFO: `/v3/projects/848c7ff5a10a47c0b98325add4101a73` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.507881 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=848c7ff5a10a47c0b98325add4101a73)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:49.510291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:49.531024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:49.554811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1256/2507] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:49.568267 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '848c7ff5a10a47c0b98325add4101a73', 'request_id': 'req-3e0935aa-a690-4f13-a8c3-c68710764c5f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.568949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc2e5e-c41e-48ea-b483-d835ed9c0fd6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:49.570186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5dfc2e5e-c41e-48ea-b483-d835ed9c0fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:49.577808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '848c7ff5a10a47c0b98325add4101a73', 'request_id': 'req-3e0935aa-a690-4f13-a8c3-c68710764c5f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.584507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '848c7ff5a10a47c0b98325add4101a73', 'request_id': 'req-3e0935aa-a690-4f13-a8c3-c68710764c5f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.595306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] The token cache is being invalidate because project 848c7ff5a10a47c0b98325add4101a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:49.595590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.596261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:49.597900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1252/2508] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:49 2022] DELETE /identity/v3/projects/848c7ff5a10a47c0b98325add4101a73 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:49.640001 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc2e5e-c41e-48ea-b483-d835ed9c0fd6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:49.697425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:49.698123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:49.698403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:49.698725 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:49.700828 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:49.774081 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:49.775570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:49.787533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1257/2509] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:28:50.133441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:50.145059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0935aa-a690-4f13-a8c3-c68710764c5f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:50.150968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a32b040b48746aa8bf040e3c98643ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b80a8c3c496d4b87b042708645c1088b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:50.151685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:50.151970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:50.152252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:50.152972 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:28:50.153311 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:50.154979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:50.169127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1253/2510] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:28:50.174484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:50.184719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a22ed-6920-4df5-a89a-4333c38c6205 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:50.190596 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f955f-2c72-4635-860c-f275150398f3 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a32b040b48746aa8bf040e3c98643ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b80a8c3c496d4b87b042708645c1088b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:50.191283 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f955f-2c72-4635-860c-f275150398f3 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:50.191573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f955f-2c72-4635-860c-f275150398f3 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:50.191865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f955f-2c72-4635-860c-f275150398f3 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:50.192578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f955f-2c72-4635-860c-f275150398f3 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:50.194804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f955f-2c72-4635-860c-f275150398f3 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:50.230133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1258/2511] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:50 2022] POST /identity/v3/projects => generated 353 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:50.231376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:50.243782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2acde-707d-4d80-a081-010fe9a1a148 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:50.296253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:50.325069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a32b040b48746aa8bf040e3c98643ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b80a8c3c496d4b87b042708645c1088b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:50.326138 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:50.326566 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:50.327109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:50.328373 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:28:50.328807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:50.331236 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:50.348849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1254/2512] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:28:50.390931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f955f-2c72-4635-860c-f275150398f3 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:50.402468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f955f-2c72-4635-860c-f275150398f3 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:50.407618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a32b040b48746aa8bf040e3c98643ef', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b80a8c3c496d4b87b042708645c1088b', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:50.408440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:50.408726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:50.409007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] PATH_INFO: `/v3/projects/8ed73e2a37274e7b9b780de4ffb93f6e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:50.410355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8ed73e2a37274e7b9b780de4ffb93f6e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:50.412523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:50.457148 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:50.474130 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ed73e2a37274e7b9b780de4ffb93f6e', 'request_id': 'req-54ee925b-0169-4882-9877-67215bf08e5e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.481036 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ed73e2a37274e7b9b780de4ffb93f6e', 'request_id': 'req-54ee925b-0169-4882-9877-67215bf08e5e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.493647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ed73e2a37274e7b9b780de4ffb93f6e', 'request_id': 'req-54ee925b-0169-4882-9877-67215bf08e5e'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.511942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] The token cache is being invalidate because project 8ed73e2a37274e7b9b780de4ffb93f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:50.512232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.512882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54ee925b-0169-4882-9877-67215bf08e5e tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.514687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1259/2513] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:50 2022] DELETE /identity/v3/projects/8ed73e2a37274e7b9b780de4ffb93f6e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:50.724709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:50.726593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:50.796651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6821322e-fc38-4354-b35f-8705df92ee67 tempest-VolumeQuotaClassesTest-227499904 tempest-VolumeQuotaClassesTest-227499904-project-admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:50.858839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:50.859729 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:50.860002 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:50.860276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] PATH_INFO: `/v3/users/65f5a71ff2ef4ec2a7ca6df1717e3b9a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:50.861957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f5a71ff2ef4ec2a7ca6df1717e3b9a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:50.863853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:50.913008 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f5a71ff2ef4ec2a7ca6df1717e3b9a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.936457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f5a71ff2ef4ec2a7ca6df1717e3b9a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.937414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f5a71ff2ef4ec2a7ca6df1717e3b9a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.947613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f5a71ff2ef4ec2a7ca6df1717e3b9a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.956758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f5a71ff2ef4ec2a7ca6df1717e3b9a'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:50.963328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1255/2514] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:50 2022] DELETE /identity/v3/users/65f5a71ff2ef4ec2a7ca6df1717e3b9a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:50.968402 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475059ac-dd5f-45f7-9584-87c1cfa333e1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:50.968734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475059ac-dd5f-45f7-9584-87c1cfa333e1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:50.969057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475059ac-dd5f-45f7-9584-87c1cfa333e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:50.988671 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-475059ac-dd5f-45f7-9584-87c1cfa333e1 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:50.990401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-475059ac-dd5f-45f7-9584-87c1cfa333e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:51.062336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1260/2515] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:51.072944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:51.074428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:51.087089 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:51.141643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0acf2-6c74-4dcb-a184-a480d2c6b42d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:51.146291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d24e05-6d42-40c7-b1a3-c0d0cf9ed993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:51.147122 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d24e05-6d42-40c7-b1a3-c0d0cf9ed993 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:51.147405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d24e05-6d42-40c7-b1a3-c0d0cf9ed993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:51.147683 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d24e05-6d42-40c7-b1a3-c0d0cf9ed993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:51.158549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d24e05-6d42-40c7-b1a3-c0d0cf9ed993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:51.159999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d24e05-6d42-40c7-b1a3-c0d0cf9ed993 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:51.170277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1256/2516] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:51.467704 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475059ac-dd5f-45f7-9584-87c1cfa333e1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:51.475251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-475059ac-dd5f-45f7-9584-87c1cfa333e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:51.491179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475059ac-dd5f-45f7-9584-87c1cfa333e1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:51.497289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:51.498201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:51.498445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:51.499371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] PATH_INFO: `/v3/projects/f8ae0c1b586b4cca8267f0e1c872d331` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:51.500395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ae0c1b586b4cca8267f0e1c872d331)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:51.502348 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:51.557907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ae0c1b586b4cca8267f0e1c872d331', 'request_id': 'req-f693beca-e090-4592-8e53-a899d94a6917'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.564134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ae0c1b586b4cca8267f0e1c872d331', 'request_id': 'req-f693beca-e090-4592-8e53-a899d94a6917'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.574833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ae0c1b586b4cca8267f0e1c872d331', 'request_id': 'req-f693beca-e090-4592-8e53-a899d94a6917'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.590740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] The token cache is being invalidate because project f8ae0c1b586b4cca8267f0e1c872d331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:51.591164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.591993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.593980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1261/2517] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:51 2022] DELETE /identity/v3/projects/f8ae0c1b586b4cca8267f0e1c872d331 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:51.598575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d24e05-6d42-40c7-b1a3-c0d0cf9ed993 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:51.612643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d24e05-6d42-40c7-b1a3-c0d0cf9ed993 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:51.676590 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:51.677843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:51.678172 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:51.678581 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] PATH_INFO: `/v3/users/4a32b040b48746aa8bf040e3c98643ef` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:51.681895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a32b040b48746aa8bf040e3c98643ef)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:51.683948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:51.735699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a32b040b48746aa8bf040e3c98643ef'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.748755 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a32b040b48746aa8bf040e3c98643ef'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.749504 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a32b040b48746aa8bf040e3c98643ef'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.757075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a32b040b48746aa8bf040e3c98643ef'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.766292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-44116be8-e5c1-45a0-a7bb-1fa24677983e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a32b040b48746aa8bf040e3c98643ef'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:51.773262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1257/2518] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:28:51 2022] DELETE /identity/v3/users/4a32b040b48746aa8bf040e3c98643ef => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:52.065304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.067136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:52.084461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693beca-e090-4592-8e53-a899d94a6917 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.128977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:52.156481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.157401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.157661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.157928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] PATH_INFO: `/v3/projects/b80a8c3c496d4b87b042708645c1088b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.159150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80a8c3c496d4b87b042708645c1088b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.161056 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.214611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80a8c3c496d4b87b042708645c1088b', 'request_id': 'req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:52.220390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80a8c3c496d4b87b042708645c1088b', 'request_id': 'req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:52.229193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80a8c3c496d4b87b042708645c1088b', 'request_id': 'req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:52.243399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] The token cache is being invalidate because project b80a8c3c496d4b87b042708645c1088b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:28:52.243675 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:52.244745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-300e9627-5f6f-48ba-b5a1-bc2996fd0cfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:28:52.246757 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1262/2519] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:28:52 2022] DELETE /identity/v3/projects/b80a8c3c496d4b87b042708645c1088b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:52.251955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba7964-c75c-4f07-82a6-4409077e0c05 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.252459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba7964-c75c-4f07-82a6-4409077e0c05 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.252459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba7964-c75c-4f07-82a6-4409077e0c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.272793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-17ba7964-c75c-4f07-82a6-4409077e0c05 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:52.274056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-17ba7964-c75c-4f07-82a6-4409077e0c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:52.301918 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:52.339750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1258/2520] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:52.350189 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc4113-67f5-4bfc-b9e9-6ba023cf30d1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.350478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc4113-67f5-4bfc-b9e9-6ba023cf30d1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.350771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc4113-67f5-4bfc-b9e9-6ba023cf30d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.371297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-62bc4113-67f5-4bfc-b9e9-6ba023cf30d1 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:52.372510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-62bc4113-67f5-4bfc-b9e9-6ba023cf30d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:52.383849 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1263/2521] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:52.388955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba7964-c75c-4f07-82a6-4409077e0c05 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.399685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba7964-c75c-4f07-82a6-4409077e0c05 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.405567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.406287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.406571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.406876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.407636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:28:52.407950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.409613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.423014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1259/2522] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:28:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:28:52.428323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc4113-67f5-4bfc-b9e9-6ba023cf30d1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.438997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc4113-67f5-4bfc-b9e9-6ba023cf30d1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.445457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301584e-be0f-4db3-b0f5-c19f9ae201a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.446143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301584e-be0f-4db3-b0f5-c19f9ae201a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.446429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301584e-be0f-4db3-b0f5-c19f9ae201a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.446705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301584e-be0f-4db3-b0f5-c19f9ae201a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.447409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301584e-be0f-4db3-b0f5-c19f9ae201a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.449383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301584e-be0f-4db3-b0f5-c19f9ae201a8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.479163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1264/2523] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:52 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:52.484194 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.496764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd4273-f9b8-433c-8b66-8b59caed23ae admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.564962 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4182e-9f2e-4410-8c43-808ebe5a9a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.565608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4182e-9f2e-4410-8c43-808ebe5a9a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.565890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4182e-9f2e-4410-8c43-808ebe5a9a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.566165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4182e-9f2e-4410-8c43-808ebe5a9a94 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.566861 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4182e-9f2e-4410-8c43-808ebe5a9a94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.569045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4182e-9f2e-4410-8c43-808ebe5a9a94 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.593342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1260/2524] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:52 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:52.596699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301584e-be0f-4db3-b0f5-c19f9ae201a8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.607436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301584e-be0f-4db3-b0f5-c19f9ae201a8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.613488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ffce-42f2-435f-bda6-66474013f533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.614166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ffce-42f2-435f-bda6-66474013f533 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.614456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ffce-42f2-435f-bda6-66474013f533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.614910 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ffce-42f2-435f-bda6-66474013f533 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.615481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796ffce-42f2-435f-bda6-66474013f533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.617112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796ffce-42f2-435f-bda6-66474013f533 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.628051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1265/2525] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:52.632406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4182e-9f2e-4410-8c43-808ebe5a9a94 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.647549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4182e-9f2e-4410-8c43-808ebe5a9a94 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.653290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae3b40-f335-4d08-aacf-ee55fbc4cfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.653994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae3b40-f335-4d08-aacf-ee55fbc4cfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.654282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae3b40-f335-4d08-aacf-ee55fbc4cfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.654564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae3b40-f335-4d08-aacf-ee55fbc4cfa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.655502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae3b40-f335-4d08-aacf-ee55fbc4cfa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.657423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae3b40-f335-4d08-aacf-ee55fbc4cfa2 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.670637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1261/2526] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:52.683050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ffce-42f2-435f-bda6-66474013f533 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.697778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ffce-42f2-435f-bda6-66474013f533 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.702659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19793838-63fc-448c-bb14-1acc47e84c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.703531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19793838-63fc-448c-bb14-1acc47e84c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.703807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19793838-63fc-448c-bb14-1acc47e84c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.704080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19793838-63fc-448c-bb14-1acc47e84c20 admin admin] PATH_INFO: `/v3/projects/ece26421b4d3448bb88aad92ded92e95/users/c993257478bb4542b00dafb36138bf40/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.720435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19793838-63fc-448c-bb14-1acc47e84c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece26421b4d3448bb88aad92ded92e95, user_id=c993257478bb4542b00dafb36138bf40, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.723113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19793838-63fc-448c-bb14-1acc47e84c20 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.738876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1266/2527] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:52 2022] PUT /identity/v3/projects/ece26421b4d3448bb88aad92ded92e95/users/c993257478bb4542b00dafb36138bf40/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:52.750996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853e1-5d68-4da4-ab29-d828f16008fa None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.751415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853e1-5d68-4da4-ab29-d828f16008fa None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.751813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab853e1-5d68-4da4-ab29-d828f16008fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.782450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-dab853e1-5d68-4da4-ab29-d828f16008fa None None] MFA Rules not processed for user `c993257478bb4542b00dafb36138bf40`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:52.783766 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dab853e1-5d68-4da4-ab29-d828f16008fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:52.852910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1262/2528] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:52.857714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19793838-63fc-448c-bb14-1acc47e84c20 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.924509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19793838-63fc-448c-bb14-1acc47e84c20 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:52.930141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679fc07-b1dd-4931-ba14-2069ba96df33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:52.930851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fc07-b1dd-4931-ba14-2069ba96df33 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:52.931161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fc07-b1dd-4931-ba14-2069ba96df33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:52.931442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679fc07-b1dd-4931-ba14-2069ba96df33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:52.932156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679fc07-b1dd-4931-ba14-2069ba96df33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:52.934336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679fc07-b1dd-4931-ba14-2069ba96df33 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:52.951705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:52.969321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1267/2529] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:52 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:52.974368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab853e1-5d68-4da4-ab29-d828f16008fa None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:52.988332 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab853e1-5d68-4da4-ab29-d828f16008fa None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.018519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:53.062401 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde7574-9e4a-422b-afaa-18905f84bc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.063083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde7574-9e4a-422b-afaa-18905f84bc21 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.063367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde7574-9e4a-422b-afaa-18905f84bc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.063645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde7574-9e4a-422b-afaa-18905f84bc21 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.064359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde7574-9e4a-422b-afaa-18905f84bc21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.066354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde7574-9e4a-422b-afaa-18905f84bc21 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.089906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1263/2530] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:52 2022] POST /identity/v3/users => generated 294 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:28:53.095635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679fc07-b1dd-4931-ba14-2069ba96df33 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.107831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679fc07-b1dd-4931-ba14-2069ba96df33 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.115970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736bf27-8156-4ebb-a462-840b48f49c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.116694 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736bf27-8156-4ebb-a462-840b48f49c7a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.116973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736bf27-8156-4ebb-a462-840b48f49c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.117277 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736bf27-8156-4ebb-a462-840b48f49c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.118030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736bf27-8156-4ebb-a462-840b48f49c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.119653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d736bf27-8156-4ebb-a462-840b48f49c7a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.131822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1268/2531] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:53.135960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde7574-9e4a-422b-afaa-18905f84bc21 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.146959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde7574-9e4a-422b-afaa-18905f84bc21 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.151954 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03b6a5-298b-469a-adcf-cb4907c3690c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.152621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03b6a5-298b-469a-adcf-cb4907c3690c admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.152908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03b6a5-298b-469a-adcf-cb4907c3690c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.153227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03b6a5-298b-469a-adcf-cb4907c3690c admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.154179 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03b6a5-298b-469a-adcf-cb4907c3690c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.155957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03b6a5-298b-469a-adcf-cb4907c3690c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.167111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1264/2532] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:53.172075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736bf27-8156-4ebb-a462-840b48f49c7a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.184257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736bf27-8156-4ebb-a462-840b48f49c7a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.189110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae1264-2ba8-4226-8d67-7b61acd9c813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.190039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae1264-2ba8-4226-8d67-7b61acd9c813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.190322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae1264-2ba8-4226-8d67-7b61acd9c813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.190601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae1264-2ba8-4226-8d67-7b61acd9c813 admin admin] PATH_INFO: `/v3/projects/b68cae1c3aa24e2f9b80f33ac5c615d4/users/aa12188e16cd4af3a762dade28b7f2b5/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.206925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae1264-2ba8-4226-8d67-7b61acd9c813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68cae1c3aa24e2f9b80f33ac5c615d4, user_id=aa12188e16cd4af3a762dade28b7f2b5, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.209738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae1264-2ba8-4226-8d67-7b61acd9c813 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.226428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1269/2533] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:53 2022] PUT /identity/v3/projects/b68cae1c3aa24e2f9b80f33ac5c615d4/users/aa12188e16cd4af3a762dade28b7f2b5/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:53.237145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793f76-9e3e-4907-81a5-c122af4fcdb6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.237525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793f76-9e3e-4907-81a5-c122af4fcdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.238742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57793f76-9e3e-4907-81a5-c122af4fcdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.268126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-57793f76-9e3e-4907-81a5-c122af4fcdb6 None None] MFA Rules not processed for user `aa12188e16cd4af3a762dade28b7f2b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:53.269685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-57793f76-9e3e-4907-81a5-c122af4fcdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:53.346016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1265/2534] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:53.352076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae1264-2ba8-4226-8d67-7b61acd9c813 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.422707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae1264-2ba8-4226-8d67-7b61acd9c813 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.428307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69ba9c-7fcb-487f-9030-8943c9b314ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.428983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69ba9c-7fcb-487f-9030-8943c9b314ef admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.429296 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69ba9c-7fcb-487f-9030-8943c9b314ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.429565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e69ba9c-7fcb-487f-9030-8943c9b314ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.430245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69ba9c-7fcb-487f-9030-8943c9b314ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.432157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e69ba9c-7fcb-487f-9030-8943c9b314ef admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.461838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1270/2535] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:28:53 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:28:53.470287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57793f76-9e3e-4907-81a5-c122af4fcdb6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.484185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57793f76-9e3e-4907-81a5-c122af4fcdb6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.547623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d99d76-9948-4586-ba2a-631c8f018844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.549303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d99d76-9948-4586-ba2a-631c8f018844 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.549303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d99d76-9948-4586-ba2a-631c8f018844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.549303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d99d76-9948-4586-ba2a-631c8f018844 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.549935 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d99d76-9948-4586-ba2a-631c8f018844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.551984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d99d76-9948-4586-ba2a-631c8f018844 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.578026 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1266/2536] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:28:53 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:28:53.579290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69ba9c-7fcb-487f-9030-8943c9b314ef admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.591530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e69ba9c-7fcb-487f-9030-8943c9b314ef admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.596610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b1316-e1aa-4c5e-a35a-c7a1de40adf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.597378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b1316-e1aa-4c5e-a35a-c7a1de40adf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.597638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b1316-e1aa-4c5e-a35a-c7a1de40adf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.597921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b1316-e1aa-4c5e-a35a-c7a1de40adf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.598665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b1316-e1aa-4c5e-a35a-c7a1de40adf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.600688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b1316-e1aa-4c5e-a35a-c7a1de40adf5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.619142 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1271/2537] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:28:53.624358 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d99d76-9948-4586-ba2a-631c8f018844 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.635623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d99d76-9948-4586-ba2a-631c8f018844 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.641697 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e3b1d-dcc4-4748-879e-438852d58b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.642787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e3b1d-dcc4-4748-879e-438852d58b5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.643069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e3b1d-dcc4-4748-879e-438852d58b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.643346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e3b1d-dcc4-4748-879e-438852d58b5e admin admin] PATH_INFO: `/v3/projects/6b58945a296141c2b69caf7ae2bce303/users/d9ad37c1fd914930ac9e3edab3cb5c9f/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.659476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e3b1d-dcc4-4748-879e-438852d58b5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b58945a296141c2b69caf7ae2bce303, user_id=d9ad37c1fd914930ac9e3edab3cb5c9f, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.662219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e3b1d-dcc4-4748-879e-438852d58b5e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.672996 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:53.679087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1267/2538] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:53 2022] PUT /identity/v3/projects/6b58945a296141c2b69caf7ae2bce303/users/d9ad37c1fd914930ac9e3edab3cb5c9f/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:53.683885 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b1316-e1aa-4c5e-a35a-c7a1de40adf5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.694636 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b1316-e1aa-4c5e-a35a-c7a1de40adf5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.777460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe2923-acb1-40bf-b7cc-ed79c8084086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.778135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2923-acb1-40bf-b7cc-ed79c8084086 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.778420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2923-acb1-40bf-b7cc-ed79c8084086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.778688 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2923-acb1-40bf-b7cc-ed79c8084086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.779446 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe2923-acb1-40bf-b7cc-ed79c8084086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.781186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe2923-acb1-40bf-b7cc-ed79c8084086 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.792318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1272/2539] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:28:53.796710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e3b1d-dcc4-4748-879e-438852d58b5e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.809338 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e3b1d-dcc4-4748-879e-438852d58b5e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:53.814562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:53.815493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.815781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.816102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] PATH_INFO: `/v3/projects/6b58945a296141c2b69caf7ae2bce303/users/d9ad37c1fd914930ac9e3edab3cb5c9f/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.818819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b58945a296141c2b69caf7ae2bce303, user_id=d9ad37c1fd914930ac9e3edab3cb5c9f, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:53.821511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:53.834749 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1268/2540] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:28:53 2022] PUT /identity/v3/projects/6b58945a296141c2b69caf7ae2bce303/users/d9ad37c1fd914930ac9e3edab3cb5c9f/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:28:53.845729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198879-b0b4-45cc-be83-38d99118ef4c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:53.846013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198879-b0b4-45cc-be83-38d99118ef4c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:53.846291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198879-b0b4-45cc-be83-38d99118ef4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:53.874435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-08198879-b0b4-45cc-be83-38d99118ef4c None None] MFA Rules not processed for user `d9ad37c1fd914930ac9e3edab3cb5c9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:28:53.875573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-08198879-b0b4-45cc-be83-38d99118ef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:53.894541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:28:53.960303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1273/2541] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:28:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:28:53.971448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:53.973531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:54.036188 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ad305-1bb9-4084-b3d8-524fa58ba013 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:54.041965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f747-0253-44f3-ab32-a66bb03e75bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:54.042814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f747-0253-44f3-ab32-a66bb03e75bd service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:54.043101 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f747-0253-44f3-ab32-a66bb03e75bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:54.043371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6099f747-0253-44f3-ab32-a66bb03e75bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:54.054305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099f747-0253-44f3-ab32-a66bb03e75bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:54.055765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6099f747-0253-44f3-ab32-a66bb03e75bd service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:54.067031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1269/2542] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:54.613930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08198879-b0b4-45cc-be83-38d99118ef4c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:54.615513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-08198879-b0b4-45cc-be83-38d99118ef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:28:54.628064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08198879-b0b4-45cc-be83-38d99118ef4c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:54.632290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4be6ff-5b13-4740-bf64-1b45b7415f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:54.632945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be6ff-5b13-4740-bf64-1b45b7415f44 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:54.633247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be6ff-5b13-4740-bf64-1b45b7415f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:54.633734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4be6ff-5b13-4740-bf64-1b45b7415f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:54.698509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4be6ff-5b13-4740-bf64-1b45b7415f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:54.700045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4be6ff-5b13-4740-bf64-1b45b7415f44 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:54.709461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1274/2543] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:28:58.857682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f747-0253-44f3-ab32-a66bb03e75bd service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:28:58.868670 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6099f747-0253-44f3-ab32-a66bb03e75bd service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:28:58.873774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db52cef-e75d-4a7d-ba13-2d8560f2ff5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:28:58.874694 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db52cef-e75d-4a7d-ba13-2d8560f2ff5e service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:28:58.874967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db52cef-e75d-4a7d-ba13-2d8560f2ff5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:28:58.875241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db52cef-e75d-4a7d-ba13-2d8560f2ff5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:28:58.944845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db52cef-e75d-4a7d-ba13-2d8560f2ff5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:28:58.944845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db52cef-e75d-4a7d-ba13-2d8560f2ff5e service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:28:58.954666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1270/2544] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:28:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:01.386629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4be6ff-5b13-4740-bf64-1b45b7415f44 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:01.460706 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4be6ff-5b13-4740-bf64-1b45b7415f44 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:01.465996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:01.466925 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:01.467204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:01.467474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] PATH_INFO: `/v3/users/c993257478bb4542b00dafb36138bf40` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:01.469191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c993257478bb4542b00dafb36138bf40)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:01.472849 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:01.529266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c993257478bb4542b00dafb36138bf40'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:01.530087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c993257478bb4542b00dafb36138bf40'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:01.530879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c993257478bb4542b00dafb36138bf40'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:01.537719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:01.547318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c993257478bb4542b00dafb36138bf40'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:01.578876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c993257478bb4542b00dafb36138bf40'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:01.585350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1275/2545] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:29:01 2022] DELETE /identity/v3/users/c993257478bb4542b00dafb36138bf40 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:01.590356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1890db-595a-41a0-baef-4a67d4fdde54 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:01.591178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1890db-595a-41a0-baef-4a67d4fdde54 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:01.591178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1890db-595a-41a0-baef-4a67d4fdde54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:01.617110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4b1890db-595a-41a0-baef-4a67d4fdde54 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:01.618475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4b1890db-595a-41a0-baef-4a67d4fdde54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:01.694443 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1271/2546] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:01.704512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:01.706082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:01.767429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d42c52-f1af-43fe-8674-c9ec48277d39 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:01.771473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab02294-6506-4802-8b1b-6ef0d4c9c806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:01.772127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab02294-6506-4802-8b1b-6ef0d4c9c806 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:01.772409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab02294-6506-4802-8b1b-6ef0d4c9c806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:01.772692 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab02294-6506-4802-8b1b-6ef0d4c9c806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:01.782570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab02294-6506-4802-8b1b-6ef0d4c9c806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:01.784010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab02294-6506-4802-8b1b-6ef0d4c9c806 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:01.794025 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1276/2547] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:02.118559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1890db-595a-41a0-baef-4a67d4fdde54 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:02.120036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4b1890db-595a-41a0-baef-4a67d4fdde54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:02.138839 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1890db-595a-41a0-baef-4a67d4fdde54 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:02.148799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:02.150752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:02.151153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:02.151499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] PATH_INFO: `/v3/projects/ece26421b4d3448bb88aad92ded92e95` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:02.152868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece26421b4d3448bb88aad92ded92e95)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:02.155112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:02.184077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:02.216695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece26421b4d3448bb88aad92ded92e95', 'request_id': 'req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.226220 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece26421b4d3448bb88aad92ded92e95', 'request_id': 'req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.235528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece26421b4d3448bb88aad92ded92e95', 'request_id': 'req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.247793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] The token cache is being invalidate because project ece26421b4d3448bb88aad92ded92e95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:29:02.248083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.248813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.250476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1272/2548] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:29:02 2022] DELETE /identity/v3/projects/ece26421b4d3448bb88aad92ded92e95 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:02.254723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab02294-6506-4802-8b1b-6ef0d4c9c806 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:02.265545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab02294-6506-4802-8b1b-6ef0d4c9c806 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:02.333643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:02.334821 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:02.335372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:02.335710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] PATH_INFO: `/v3/users/aa12188e16cd4af3a762dade28b7f2b5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:02.336974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa12188e16cd4af3a762dade28b7f2b5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:02.339027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:02.405906 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa12188e16cd4af3a762dade28b7f2b5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.406835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa12188e16cd4af3a762dade28b7f2b5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.407546 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa12188e16cd4af3a762dade28b7f2b5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.430201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa12188e16cd4af3a762dade28b7f2b5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.440871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa12188e16cd4af3a762dade28b7f2b5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:02.448134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:02.470355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1277/2549] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:29:02 2022] DELETE /identity/v3/users/aa12188e16cd4af3a762dade28b7f2b5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:02.866963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:02.868442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:02.885461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a9530-19cd-467a-ae1d-fc5c0ac53ec3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:02.947444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:02.948321 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:02.948603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:02.948877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] PATH_INFO: `/v3/projects/b68cae1c3aa24e2f9b80f33ac5c615d4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:02.950143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68cae1c3aa24e2f9b80f33ac5c615d4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:02.952038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:03.003752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68cae1c3aa24e2f9b80f33ac5c615d4', 'request_id': 'req-a79e2791-cec6-44fc-860a-88a6507cc016'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.014259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68cae1c3aa24e2f9b80f33ac5c615d4', 'request_id': 'req-a79e2791-cec6-44fc-860a-88a6507cc016'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.020950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68cae1c3aa24e2f9b80f33ac5c615d4', 'request_id': 'req-a79e2791-cec6-44fc-860a-88a6507cc016'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.034361 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] The token cache is being invalidate because project b68cae1c3aa24e2f9b80f33ac5c615d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:29:03.034649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.035340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.037535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1273/2550] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:29:02 2022] DELETE /identity/v3/projects/b68cae1c3aa24e2f9b80f33ac5c615d4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:03.041589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:03.043002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:03.056295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deba5ec-effa-42c9-9ac1-52b45926c21d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:03.116159 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:03.117053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:03.117364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:03.117643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] PATH_INFO: `/v3/users/d9ad37c1fd914930ac9e3edab3cb5c9f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:03.118842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9ad37c1fd914930ac9e3edab3cb5c9f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:03.120893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:03.173818 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9ad37c1fd914930ac9e3edab3cb5c9f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.174474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9ad37c1fd914930ac9e3edab3cb5c9f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.175076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9ad37c1fd914930ac9e3edab3cb5c9f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.188256 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9ad37c1fd914930ac9e3edab3cb5c9f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.197407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-036abe31-6dac-4057-9f43-f7fad671aa24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9ad37c1fd914930ac9e3edab3cb5c9f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.210902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1278/2551] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:29:03 2022] DELETE /identity/v3/users/d9ad37c1fd914930ac9e3edab3cb5c9f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:03.510077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:03.511635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:03.526679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e2791-cec6-44fc-860a-88a6507cc016 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:03.572955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:03.591737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:03.592644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:03.592934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:03.593246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] PATH_INFO: `/v3/projects/6b58945a296141c2b69caf7ae2bce303` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:03.594496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b58945a296141c2b69caf7ae2bce303)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:03.596406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:03.649430 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b58945a296141c2b69caf7ae2bce303', 'request_id': 'req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.658882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b58945a296141c2b69caf7ae2bce303', 'request_id': 'req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.665843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b58945a296141c2b69caf7ae2bce303', 'request_id': 'req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.676963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] The token cache is being invalidate because project 6b58945a296141c2b69caf7ae2bce303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:29:03.677406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.678187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-345bcdb8-bff6-40a3-bcb1-08469d1b6aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:03.679916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1274/2552] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:29:03 2022] DELETE /identity/v3/projects/6b58945a296141c2b69caf7ae2bce303 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:03.686254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329c96-99aa-404c-a109-5a1691d2f466 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:03.686998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329c96-99aa-404c-a109-5a1691d2f466 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:03.686998 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75329c96-99aa-404c-a109-5a1691d2f466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:03.707192 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-75329c96-99aa-404c-a109-5a1691d2f466 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:03.708730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-75329c96-99aa-404c-a109-5a1691d2f466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:03.774762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1279/2553] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:29:03.783920 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56945211-35ff-48db-b742-19065d760bee None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:03.784212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56945211-35ff-48db-b742-19065d760bee None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:03.784488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56945211-35ff-48db-b742-19065d760bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:03.805836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-56945211-35ff-48db-b742-19065d760bee None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:03.807155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-56945211-35ff-48db-b742-19065d760bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:03.816156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1275/2554] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:03.821314 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75329c96-99aa-404c-a109-5a1691d2f466 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:03.831534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75329c96-99aa-404c-a109-5a1691d2f466 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:03.837094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:03.837798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:03.838087 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:03.838360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:03.839093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:03.839405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:03.841406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:03.851123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:03.858419 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1280/2555] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:03.859463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56945211-35ff-48db-b742-19065d760bee None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:03.869785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56945211-35ff-48db-b742-19065d760bee None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:03.875004 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcfb2a-e1ff-471f-b496-9312fb6c87e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:03.875674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfb2a-e1ff-471f-b496-9312fb6c87e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:03.875955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfb2a-e1ff-471f-b496-9312fb6c87e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:03.876227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfb2a-e1ff-471f-b496-9312fb6c87e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:03.876940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcfb2a-e1ff-471f-b496-9312fb6c87e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:03.878939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcfb2a-e1ff-471f-b496-9312fb6c87e8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:03.909071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1276/2556] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:29:03 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:29:03.913533 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:03.924193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdfc2e-1d91-41f1-98c3-50fa35144251 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:03.984517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b182c-ddc1-4237-abbf-09013b570b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:03.985220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b182c-ddc1-4237-abbf-09013b570b44 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:03.985522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b182c-ddc1-4237-abbf-09013b570b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:03.985802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b182c-ddc1-4237-abbf-09013b570b44 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:03.986808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b182c-ddc1-4237-abbf-09013b570b44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:03.988721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b182c-ddc1-4237-abbf-09013b570b44 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:04.010133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1281/2557] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:29:03 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:04.014575 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcfb2a-e1ff-471f-b496-9312fb6c87e8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:04.025293 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcfb2a-e1ff-471f-b496-9312fb6c87e8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:04.030333 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3abcaa-d298-49b7-ad6e-1afcc6447bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:04.030998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abcaa-d298-49b7-ad6e-1afcc6447bcc admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:04.031343 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abcaa-d298-49b7-ad6e-1afcc6447bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:04.031623 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3abcaa-d298-49b7-ad6e-1afcc6447bcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:04.032378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3abcaa-d298-49b7-ad6e-1afcc6447bcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:04.034033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3abcaa-d298-49b7-ad6e-1afcc6447bcc admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:04.047452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1277/2558] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:29:04.049293 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b182c-ddc1-4237-abbf-09013b570b44 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:04.059672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b182c-ddc1-4237-abbf-09013b570b44 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:04.065612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf4dd2-761d-490e-b183-727485a5884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:04.066312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf4dd2-761d-490e-b183-727485a5884f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:04.066594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf4dd2-761d-490e-b183-727485a5884f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:04.066871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf4dd2-761d-490e-b183-727485a5884f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:04.067609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf4dd2-761d-490e-b183-727485a5884f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:04.069337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf4dd2-761d-490e-b183-727485a5884f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:04.080282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1282/2559] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:29:04.085128 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3abcaa-d298-49b7-ad6e-1afcc6447bcc admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:04.095657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3abcaa-d298-49b7-ad6e-1afcc6447bcc admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:04.100563 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:04.101480 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:04.101765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:04.102045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] PATH_INFO: `/v3/projects/61b68ce75b4846ef9ef3a169cbaa4f8d/users/db132f0e70484abeba0c2399c6fa3ece/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:04.118134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61b68ce75b4846ef9ef3a169cbaa4f8d, user_id=db132f0e70484abeba0c2399c6fa3ece, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:04.120811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:04.135351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1278/2560] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:29:04 2022] PUT /identity/v3/projects/61b68ce75b4846ef9ef3a169cbaa4f8d/users/db132f0e70484abeba0c2399c6fa3ece/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:29:04.147308 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2054c8b-846e-4184-a274-b7902b6c2c99 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:04.147578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2054c8b-846e-4184-a274-b7902b6c2c99 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:04.147894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2054c8b-846e-4184-a274-b7902b6c2c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:04.176320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e2054c8b-846e-4184-a274-b7902b6c2c99 None None] MFA Rules not processed for user `db132f0e70484abeba0c2399c6fa3ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:04.179054 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e2054c8b-846e-4184-a274-b7902b6c2c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:04.250477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1283/2561] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:04.261487 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:04.263376 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:04.323470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3884175-3726-4bb4-af22-00b8e9c3d801 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:04.328203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06fb296-65fa-40a0-944d-4dbdb0ad8026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:04.328887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fb296-65fa-40a0-944d-4dbdb0ad8026 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:04.329193 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fb296-65fa-40a0-944d-4dbdb0ad8026 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:04.329474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06fb296-65fa-40a0-944d-4dbdb0ad8026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:04.339959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fb296-65fa-40a0-944d-4dbdb0ad8026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:04.341631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06fb296-65fa-40a0-944d-4dbdb0ad8026 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:04.351543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1279/2562] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:06.891088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2054c8b-846e-4184-a274-b7902b6c2c99 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:06.892891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e2054c8b-846e-4184-a274-b7902b6c2c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:06.999447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2054c8b-846e-4184-a274-b7902b6c2c99 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:07.006405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:07.007355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:07.007817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:07.008242 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:07.011252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:07.022082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:07.028184 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:07.031090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:07.046583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1284/2563] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:07.169743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad639-53f6-4821-bd17-0baa2f602ef9 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:07.170774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad639-53f6-4821-bd17-0baa2f602ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:07.170774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ad639-53f6-4821-bd17-0baa2f602ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:07.194434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b09ad639-53f6-4821-bd17-0baa2f602ef9 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:07.195911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b09ad639-53f6-4821-bd17-0baa2f602ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:07.203412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:07.280800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1280/2564] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:29:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:07.293702 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:07.295297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:07.380570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b2fb4-edc1-49f8-a950-3e32ab77d112 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:07.386009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb9b4d-b25f-4d26-8293-d88e90dd7af1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:07.386699 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb9b4d-b25f-4d26-8293-d88e90dd7af1 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:07.387035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb9b4d-b25f-4d26-8293-d88e90dd7af1 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:07.387270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb9b4d-b25f-4d26-8293-d88e90dd7af1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:07.396626 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb9b4d-b25f-4d26-8293-d88e90dd7af1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:07.398320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb9b4d-b25f-4d26-8293-d88e90dd7af1 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:07.409784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1285/2565] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:29:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:29:15.367536 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad639-53f6-4821-bd17-0baa2f602ef9 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:15.497896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ad639-53f6-4821-bd17-0baa2f602ef9 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:15.504938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:15.508209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:15.508593 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:15.513750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] PATH_INFO: `/v3/users/db132f0e70484abeba0c2399c6fa3ece` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:15.517466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db132f0e70484abeba0c2399c6fa3ece)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:15.524227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:15.671192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db132f0e70484abeba0c2399c6fa3ece'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:15.693262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db132f0e70484abeba0c2399c6fa3ece'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:15.694728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db132f0e70484abeba0c2399c6fa3ece'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:15.716619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db132f0e70484abeba0c2399c6fa3ece'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:15.734393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db132f0e70484abeba0c2399c6fa3ece'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:15.744814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1281/2566] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:29:15 2022] DELETE /identity/v3/users/db132f0e70484abeba0c2399c6fa3ece => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:15.750155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c03bdd-94d5-4702-8cf6-213af9dc6a73 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:15.753230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c03bdd-94d5-4702-8cf6-213af9dc6a73 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:15.753553 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c03bdd-94d5-4702-8cf6-213af9dc6a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:15.774560 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-30c03bdd-94d5-4702-8cf6-213af9dc6a73 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:15.776021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-30c03bdd-94d5-4702-8cf6-213af9dc6a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:15.871279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1286/2567] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:15.887896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:15.890675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:15.973429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:16.000677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a5979-3490-4874-a92b-1a9bf4e66e40 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:16.004577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29e4a-9a00-4ec8-9c8e-3beb33dff9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:16.006369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29e4a-9a00-4ec8-9c8e-3beb33dff9fb service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:16.006369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29e4a-9a00-4ec8-9c8e-3beb33dff9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:16.006369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29e4a-9a00-4ec8-9c8e-3beb33dff9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:16.019899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a29e4a-9a00-4ec8-9c8e-3beb33dff9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:16.023785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a29e4a-9a00-4ec8-9c8e-3beb33dff9fb service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:16.038363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1282/2568] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:16.644442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c03bdd-94d5-4702-8cf6-213af9dc6a73 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:16.646199 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-30c03bdd-94d5-4702-8cf6-213af9dc6a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:16.668583 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c03bdd-94d5-4702-8cf6-213af9dc6a73 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:16.696954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:16.698773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:16.699059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:16.699338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] PATH_INFO: `/v3/projects/61b68ce75b4846ef9ef3a169cbaa4f8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:16.703438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=61b68ce75b4846ef9ef3a169cbaa4f8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:16.710484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:16.794381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61b68ce75b4846ef9ef3a169cbaa4f8d', 'request_id': 'req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:16.803191 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61b68ce75b4846ef9ef3a169cbaa4f8d', 'request_id': 'req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:16.819516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61b68ce75b4846ef9ef3a169cbaa4f8d', 'request_id': 'req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:16.837639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] The token cache is being invalidate because project 61b68ce75b4846ef9ef3a169cbaa4f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:29:16.838171 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:16.838848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7a1b6c22-3e61-44ae-ab89-e28b8ca82f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:16.844758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1287/2569] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:29:16 2022] DELETE /identity/v3/projects/61b68ce75b4846ef9ef3a169cbaa4f8d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:16.846591 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96a332-2c0e-4011-9091-b28e55ed32cd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:16.847093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96a332-2c0e-4011-9091-b28e55ed32cd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:16.847541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96a332-2c0e-4011-9091-b28e55ed32cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:16.871045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7a96a332-2c0e-4011-9091-b28e55ed32cd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:16.872242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7a96a332-2c0e-4011-9091-b28e55ed32cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:16.961339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1283/2570] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:16.969516 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba4164-e674-425d-b153-9fe410b87d3e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:16.970144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba4164-e674-425d-b153-9fe410b87d3e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:16.970536 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ba4164-e674-425d-b153-9fe410b87d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:16.986300 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:17.014140 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c1ba4164-e674-425d-b153-9fe410b87d3e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:17.020076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c1ba4164-e674-425d-b153-9fe410b87d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:17.062502 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1288/2571] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:17.066577 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96a332-2c0e-4011-9091-b28e55ed32cd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.088774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96a332-2c0e-4011-9091-b28e55ed32cd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:17.123452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:17.124156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.124442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.124739 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.125625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:17.135561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:17.139666 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:17.174201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1284/2572] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:17.175465 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba4164-e674-425d-b153-9fe410b87d3e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.192105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ba4164-e674-425d-b153-9fe410b87d3e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:17.197952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cf9ea-c618-4797-a9f0-d5c9be42a4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:17.199133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cf9ea-c618-4797-a9f0-d5c9be42a4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.199970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cf9ea-c618-4797-a9f0-d5c9be42a4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.200558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cf9ea-c618-4797-a9f0-d5c9be42a4d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.201618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cf9ea-c618-4797-a9f0-d5c9be42a4d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:17.204304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cf9ea-c618-4797-a9f0-d5c9be42a4d9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:17.269207 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1289/2573] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:29:17 2022] POST /identity/v3/projects => generated 397 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:17.328834 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.341920 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da3915-9601-47fa-8d9d-79bc6165e882 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:17.477134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46b657-fee7-441c-aa3d-1d89e35a4ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:17.477952 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b657-fee7-441c-aa3d-1d89e35a4ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.478231 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b657-fee7-441c-aa3d-1d89e35a4ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.478512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b657-fee7-441c-aa3d-1d89e35a4ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.479615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46b657-fee7-441c-aa3d-1d89e35a4ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:17.481607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46b657-fee7-441c-aa3d-1d89e35a4ea4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:17.518084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1285/2574] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:29:17 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:17.522734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cf9ea-c618-4797-a9f0-d5c9be42a4d9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.538075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cf9ea-c618-4797-a9f0-d5c9be42a4d9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:17.551510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f9214-f495-4cb6-a545-6afc4e5910e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:17.552839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f9214-f495-4cb6-a545-6afc4e5910e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.553867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f9214-f495-4cb6-a545-6afc4e5910e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.553867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f9214-f495-4cb6-a545-6afc4e5910e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.554235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f9214-f495-4cb6-a545-6afc4e5910e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:17.555940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f9214-f495-4cb6-a545-6afc4e5910e0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:17.569836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1290/2575] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:29:17.574784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46b657-fee7-441c-aa3d-1d89e35a4ea4 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.587641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46b657-fee7-441c-aa3d-1d89e35a4ea4 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:17.595820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2f1ee-a929-480d-a357-0a29c05ecc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:17.596518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2f1ee-a929-480d-a357-0a29c05ecc83 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.598092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2f1ee-a929-480d-a357-0a29c05ecc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.598092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2f1ee-a929-480d-a357-0a29c05ecc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.598092 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2f1ee-a929-480d-a357-0a29c05ecc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:17.599775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2f1ee-a929-480d-a357-0a29c05ecc83 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:17.612303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1286/2576] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:29:17.617992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f9214-f495-4cb6-a545-6afc4e5910e0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.637404 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f9214-f495-4cb6-a545-6afc4e5910e0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:17.644086 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:17.645543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.659119 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.659817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] PATH_INFO: `/v3/projects/cbd2d870adac48849df1ba578d6bc3ce/users/330fc8aa56ca4aa48bd300b0176b3155/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.701752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbd2d870adac48849df1ba578d6bc3ce, user_id=330fc8aa56ca4aa48bd300b0176b3155, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:17.704841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:17.748089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1291/2577] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:29:17 2022] PUT /identity/v3/projects/cbd2d870adac48849df1ba578d6bc3ce/users/330fc8aa56ca4aa48bd300b0176b3155/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:17.758553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39524250-77e9-496a-8e09-28a6ace8759c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:17.758847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39524250-77e9-496a-8e09-28a6ace8759c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:17.759133 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39524250-77e9-496a-8e09-28a6ace8759c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:17.791213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-39524250-77e9-496a-8e09-28a6ace8759c None None] MFA Rules not processed for user `330fc8aa56ca4aa48bd300b0176b3155`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:17.794259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-39524250-77e9-496a-8e09-28a6ace8759c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:17.862823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:17.969914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1287/2578] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:17.989163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:17.991602 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:18.131513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e3ddc8-fd64-4087-89be-2086c0d8f622 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:18.136863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378ea43-ca21-4fce-b604-1085f553983c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:18.137794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ea43-ca21-4fce-b604-1085f553983c service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:18.138073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ea43-ca21-4fce-b604-1085f553983c service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:18.138723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378ea43-ca21-4fce-b604-1085f553983c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:18.150924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378ea43-ca21-4fce-b604-1085f553983c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:18.152011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378ea43-ca21-4fce-b604-1085f553983c service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:18.164064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1292/2579] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:19.130450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c19246-7993-4a7e-98a9-abe12f2ff8f1 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:19.131791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c19246-7993-4a7e-98a9-abe12f2ff8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:19.131791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c19246-7993-4a7e-98a9-abe12f2ff8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:19.157460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-25c19246-7993-4a7e-98a9-abe12f2ff8f1 None None] MFA Rules not processed for user `fbbafeb56a4d4fd282275e4198ede783`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:19.158819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-25c19246-7993-4a7e-98a9-abe12f2ff8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:19.229819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1288/2580] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:25.269230 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7d56e-5a6f-4d40-80e9-ac20508c3ccd None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.269941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7d56e-5a6f-4d40-80e9-ac20508c3ccd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.270444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca7d56e-5a6f-4d40-80e9-ac20508c3ccd None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.273229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1293/2581] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:29:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 17:29:25.281806 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b70d21-73a5-4e4b-909f-8bebeb2064bd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.282347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b70d21-73a5-4e4b-909f-8bebeb2064bd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.282347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b70d21-73a5-4e4b-909f-8bebeb2064bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.306137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d4b70d21-73a5-4e4b-909f-8bebeb2064bd None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:25.307926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d4b70d21-73a5-4e4b-909f-8bebeb2064bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:25.407303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1289/2582] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:29:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:29:25.424824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7d56e-5a6f-4d40-80e9-ac20508c3ccd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:25.427332 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fca7d56e-5a6f-4d40-80e9-ac20508c3ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:25.531177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca7d56e-5a6f-4d40-80e9-ac20508c3ccd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:25.537463 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336b807a-77d5-48f1-acc6-6ab68d0bef32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:25.538642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b807a-77d5-48f1-acc6-6ab68d0bef32 service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.539202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b807a-77d5-48f1-acc6-6ab68d0bef32 service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.539679 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b807a-77d5-48f1-acc6-6ab68d0bef32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.556495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b807a-77d5-48f1-acc6-6ab68d0bef32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:25.558381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336b807a-77d5-48f1-acc6-6ab68d0bef32 service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:25.574562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1294/2583] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:25.576652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b70d21-73a5-4e4b-909f-8bebeb2064bd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:25.678622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c44ea-e762-499e-a0f6-f753c3de519a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.679470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c44ea-e762-499e-a0f6-f753c3de519a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.679968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c44ea-e762-499e-a0f6-f753c3de519a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.680550 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b70d21-73a5-4e4b-909f-8bebeb2064bd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:25.682820 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1295/2584] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:29:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:29:25.688103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569749a-10a3-499c-8322-2c5a4dc4ad2a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.688103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569749a-10a3-499c-8322-2c5a4dc4ad2a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.688380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1569749a-10a3-499c-8322-2c5a4dc4ad2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.691583 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:25.693964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.695083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.695944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] PATH_INFO: `/v3/users/330fc8aa56ca4aa48bd300b0176b3155` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.698559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] RBAC: Authorizing `identity:delete_user(user_id=330fc8aa56ca4aa48bd300b0176b3155)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:25.703337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:25.710595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1569749a-10a3-499c-8322-2c5a4dc4ad2a None None] MFA Rules not processed for user `72fb6996e1974060b162070f8da365aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:25.712563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1569749a-10a3-499c-8322-2c5a4dc4ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:25.738291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:25.773729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:25.807649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '330fc8aa56ca4aa48bd300b0176b3155'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:25.816133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1296/2585] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:29:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:29:25.829257 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1569749a-10a3-499c-8322-2c5a4dc4ad2a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:25.831936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1569749a-10a3-499c-8322-2c5a4dc4ad2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:25.832735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '330fc8aa56ca4aa48bd300b0176b3155'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:25.834297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '330fc8aa56ca4aa48bd300b0176b3155'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:25.847049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '330fc8aa56ca4aa48bd300b0176b3155'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:25.847730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1569749a-10a3-499c-8322-2c5a4dc4ad2a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:25.853983 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a713d5-0611-40ac-9865-be7968932f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:25.855035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a713d5-0611-40ac-9865-be7968932f6d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.855338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a713d5-0611-40ac-9865-be7968932f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.855807 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a713d5-0611-40ac-9865-be7968932f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.858685 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d6a713d5-0611-40ac-9865-be7968932f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:25.861686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-5ede259a-ef62-4050-8cc3-079212814978 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '330fc8aa56ca4aa48bd300b0176b3155'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:25.876917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a713d5-0611-40ac-9865-be7968932f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:25.877858 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1290/2586] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:29:25 2022] DELETE /identity/v3/users/330fc8aa56ca4aa48bd300b0176b3155 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:29:25.878670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a713d5-0611-40ac-9865-be7968932f6d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:25.884726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a918d-d631-417b-b545-3febcf5a5179 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:25.885435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a918d-d631-417b-b545-3febcf5a5179 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:25.888111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a918d-d631-417b-b545-3febcf5a5179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:25.890805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1297/2587] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:25.921150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f23a918d-d631-417b-b545-3febcf5a5179 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:25.922415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f23a918d-d631-417b-b545-3febcf5a5179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:26.017782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5a2d-447f-408d-9a7d-10ef3eb03231 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.018069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5a2d-447f-408d-9a7d-10ef3eb03231 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.018353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5a2d-447f-408d-9a7d-10ef3eb03231 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.020333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1298/2588] 10.222.0.66 () {54 vars in 921 bytes} [Thu Jun 16 17:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 17:29:26.029619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1291/2589] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:29:26.038844 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623954b-a2cc-43d2-ae30-fcef328cb7d4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.040138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623954b-a2cc-43d2-ae30-fcef328cb7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.040445 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623954b-a2cc-43d2-ae30-fcef328cb7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.046710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a918d-d631-417b-b545-3febcf5a5179 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.049013 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f23a918d-d631-417b-b545-3febcf5a5179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:26.073857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0623954b-a2cc-43d2-ae30-fcef328cb7d4 None None] MFA Rules not processed for user `e3241452a88a40da9c9b8fa79c4fc9c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:26.089450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0623954b-a2cc-43d2-ae30-fcef328cb7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:26.201078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a918d-d631-417b-b545-3febcf5a5179 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.205970 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:26.206908 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.207603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.208170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.220046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:26.222241 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:26.237273 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1292/2590] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:29:26.258819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1299/2591] 10.222.0.66 () {58 vars in 1018 bytes} [Thu Jun 16 17:29:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:26.278681 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.280993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:26.304956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393d342-86c0-4968-955f-33ac67c360df service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.315069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39e2c8-b6e9-4735-b60f-d3fb83e89db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:26.315592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e2c8-b6e9-4735-b60f-d3fb83e89db9 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:26.315940 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e2c8-b6e9-4735-b60f-d3fb83e89db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:26.316296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39e2c8-b6e9-4735-b60f-d3fb83e89db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:26.331771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39e2c8-b6e9-4735-b60f-d3fb83e89db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:26.333249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39e2c8-b6e9-4735-b60f-d3fb83e89db9 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:26.351308 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1293/2592] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:26.948659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623954b-a2cc-43d2-ae30-fcef328cb7d4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:26.950587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0623954b-a2cc-43d2-ae30-fcef328cb7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:26.988631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623954b-a2cc-43d2-ae30-fcef328cb7d4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:26.995904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:26.997759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.006258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.006502 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] PATH_INFO: `/v3/projects/cbd2d870adac48849df1ba578d6bc3ce` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.008543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbd2d870adac48849df1ba578d6bc3ce)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.011263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.124507 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbd2d870adac48849df1ba578d6bc3ce', 'request_id': 'req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:27.139903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbd2d870adac48849df1ba578d6bc3ce', 'request_id': 'req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:27.155993 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbd2d870adac48849df1ba578d6bc3ce', 'request_id': 'req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:27.183439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] The token cache is being invalidate because project cbd2d870adac48849df1ba578d6bc3ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:29:27.183922 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:27.185561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f107b1a1-ef76-40fc-a99a-7a548c43cd2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:27.199155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1300/2593] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:29:26 2022] DELETE /identity/v3/projects/cbd2d870adac48849df1ba578d6bc3ce => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:27.206521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b9385-a632-4986-9f4b-dddf74c5be9a None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.206805 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b9385-a632-4986-9f4b-dddf74c5be9a None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.207081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b9385-a632-4986-9f4b-dddf74c5be9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.250909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-349b9385-a632-4986-9f4b-dddf74c5be9a None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:27.253462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-349b9385-a632-4986-9f4b-dddf74c5be9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:27.287849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:27.377609 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1294/2594] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:29:27.387174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a8975-7cc1-479f-9e85-78ad75390137 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.387527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a8975-7cc1-479f-9e85-78ad75390137 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.389689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a8975-7cc1-479f-9e85-78ad75390137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.414057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-de9a8975-7cc1-479f-9e85-78ad75390137 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:27.416079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-de9a8975-7cc1-479f-9e85-78ad75390137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:27.426865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1301/2595] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:27.432631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b9385-a632-4986-9f4b-dddf74c5be9a None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.444870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b9385-a632-4986-9f4b-dddf74c5be9a None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.453842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.455624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.455898 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.456184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.458034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:27.458966 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.460819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.480287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1295/2596] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:27.485556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a8975-7cc1-479f-9e85-78ad75390137 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.499316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a8975-7cc1-479f-9e85-78ad75390137 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.506767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c5138-50b6-4eb8-afa3-2280942b7e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.507532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5138-50b6-4eb8-afa3-2280942b7e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.507876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5138-50b6-4eb8-afa3-2280942b7e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.508156 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5138-50b6-4eb8-afa3-2280942b7e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.508952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c5138-50b6-4eb8-afa3-2280942b7e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.512935 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c5138-50b6-4eb8-afa3-2280942b7e9d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.561862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1302/2597] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:29:27.563149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.574427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407a4e5-a748-4c3a-bcd0-e9c72e2e0378 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.662352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08869d8f-1c23-4383-852a-160088fb4d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.663502 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08869d8f-1c23-4383-852a-160088fb4d4c admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.663690 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08869d8f-1c23-4383-852a-160088fb4d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.663973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08869d8f-1c23-4383-852a-160088fb4d4c admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.664701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08869d8f-1c23-4383-852a-160088fb4d4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.666751 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08869d8f-1c23-4383-852a-160088fb4d4c admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.691633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1296/2598] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:29:27.697819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c5138-50b6-4eb8-afa3-2280942b7e9d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.710783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c5138-50b6-4eb8-afa3-2280942b7e9d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.715930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796279-f634-42e5-b6d6-ace337571df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.716650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796279-f634-42e5-b6d6-ace337571df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.716933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796279-f634-42e5-b6d6-ace337571df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.717249 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44796279-f634-42e5-b6d6-ace337571df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.717990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796279-f634-42e5-b6d6-ace337571df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.719632 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44796279-f634-42e5-b6d6-ace337571df7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.733232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1303/2599] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:29:27.737476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08869d8f-1c23-4383-852a-160088fb4d4c admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.754228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08869d8f-1c23-4383-852a-160088fb4d4c admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.774598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57dac5-f742-43c3-942b-9a2da9b43cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.775427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57dac5-f742-43c3-942b-9a2da9b43cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.775765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57dac5-f742-43c3-942b-9a2da9b43cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.776104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57dac5-f742-43c3-942b-9a2da9b43cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.777474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57dac5-f742-43c3-942b-9a2da9b43cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.780761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57dac5-f742-43c3-942b-9a2da9b43cf4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.803871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796279-f634-42e5-b6d6-ace337571df7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:27.805405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1297/2600] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:29:27.820088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44796279-f634-42e5-b6d6-ace337571df7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:27.832505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:27.834742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.835180 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.835769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] PATH_INFO: `/v3/projects/50aeadccaa9348fd9af173f94291bf6f/users/6cd1d0ce941845c3a28e2cd5fc77c844/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.842392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:27.865743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50aeadccaa9348fd9af173f94291bf6f, user_id=6cd1d0ce941845c3a28e2cd5fc77c844, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:27.869848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:27.892169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1304/2601] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:29:27 2022] PUT /identity/v3/projects/50aeadccaa9348fd9af173f94291bf6f/users/6cd1d0ce941845c3a28e2cd5fc77c844/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:27.899854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398191d8-3668-49ba-bc53-f7a2f0984a2e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:27.900252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398191d8-3668-49ba-bc53-f7a2f0984a2e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:27.900646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398191d8-3668-49ba-bc53-f7a2f0984a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:27.928134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-398191d8-3668-49ba-bc53-f7a2f0984a2e None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:27.929910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-398191d8-3668-49ba-bc53-f7a2f0984a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:28.028527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1298/2602] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:29:28.047401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:28.050945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:28.177717 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64f56d-4c15-4c76-ba3f-0e7e4114f214 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:28.185710 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:28.186590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:28.187049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:28.187335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:28.213654 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:28.216303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:28.237004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1305/2603] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:31.515544 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216610e-1480-4312-9899-5e4ee1b63be4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:31.516081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216610e-1480-4312-9899-5e4ee1b63be4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:31.516201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216610e-1480-4312-9899-5e4ee1b63be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:31.544496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6216610e-1480-4312-9899-5e4ee1b63be4 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:31.545855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6216610e-1480-4312-9899-5e4ee1b63be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:31.601009 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:31.683140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1299/2604] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:29:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:31.696948 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:31.699545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:31.829616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e7b32-3c5b-49a2-a332-185e2fdd9efc service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:31.835360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a090f-bba8-4c79-b340-3191ddb9ed00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:31.836048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a090f-bba8-4c79-b340-3191ddb9ed00 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:31.836339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a090f-bba8-4c79-b340-3191ddb9ed00 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:31.836618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a090f-bba8-4c79-b340-3191ddb9ed00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:31.849563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a090f-bba8-4c79-b340-3191ddb9ed00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:31.851528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a090f-bba8-4c79-b340-3191ddb9ed00 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:31.865352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1306/2605] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:29:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:29:37.644442 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10b4a8-564a-4ce1-a4b5-4d495cb63f9b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.646253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10b4a8-564a-4ce1-a4b5-4d495cb63f9b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.647823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10b4a8-564a-4ce1-a4b5-4d495cb63f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.709521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2f10b4a8-564a-4ce1-a4b5-4d495cb63f9b None None] MFA Rules not processed for user `6cd1d0ce941845c3a28e2cd5fc77c844`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:37.713275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2f10b4a8-564a-4ce1-a4b5-4d495cb63f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:37.931125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1300/2606] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:37.935744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a090f-bba8-4c79-b340-3191ddb9ed00 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:37.950589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a090f-bba8-4c79-b340-3191ddb9ed00 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:37.963852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1b8f5-ca46-42fd-8f94-8e7352cf7522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:37.964662 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b8f5-ca46-42fd-8f94-8e7352cf7522 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:37.965154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b8f5-ca46-42fd-8f94-8e7352cf7522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:37.965570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1b8f5-ca46-42fd-8f94-8e7352cf7522 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:37.966358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1b8f5-ca46-42fd-8f94-8e7352cf7522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:37.969749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1b8f5-ca46-42fd-8f94-8e7352cf7522 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.050226 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1307/2607] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:29:37 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:38.054720 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10b4a8-564a-4ce1-a4b5-4d495cb63f9b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.074580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10b4a8-564a-4ce1-a4b5-4d495cb63f9b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.273229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36f820-a93b-4763-b599-88f581b55ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.274673 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f820-a93b-4763-b599-88f581b55ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.275216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f820-a93b-4763-b599-88f581b55ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.275846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f820-a93b-4763-b599-88f581b55ffa admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.276955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36f820-a93b-4763-b599-88f581b55ffa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.280227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36f820-a93b-4763-b599-88f581b55ffa admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.327476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1301/2608] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:29:38 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:38.328848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1b8f5-ca46-42fd-8f94-8e7352cf7522 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.348207 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1b8f5-ca46-42fd-8f94-8e7352cf7522 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.359742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324345-c1e5-46da-ac1e-832e268fbd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.362118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324345-c1e5-46da-ac1e-832e268fbd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.362858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324345-c1e5-46da-ac1e-832e268fbd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.363550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27324345-c1e5-46da-ac1e-832e268fbd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.369029 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27324345-c1e5-46da-ac1e-832e268fbd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.373313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27324345-c1e5-46da-ac1e-832e268fbd92 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.414381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1308/2609] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:29:38.415991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36f820-a93b-4763-b599-88f581b55ffa admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.435230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36f820-a93b-4763-b599-88f581b55ffa admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.447923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376965d-9a1e-4e0a-afa4-05f8057136f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.449414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376965d-9a1e-4e0a-afa4-05f8057136f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.449852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376965d-9a1e-4e0a-afa4-05f8057136f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.450235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376965d-9a1e-4e0a-afa4-05f8057136f0 admin admin] PATH_INFO: `/v3/projects/8612326dfc09414caca0b216d53b9f13/users/ff1b16ee76d34b72b236b28dfbd94fb8/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.479752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376965d-9a1e-4e0a-afa4-05f8057136f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8612326dfc09414caca0b216d53b9f13, user_id=ff1b16ee76d34b72b236b28dfbd94fb8, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.482837 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376965d-9a1e-4e0a-afa4-05f8057136f0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.507484 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1302/2610] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:29:38 2022] PUT /identity/v3/projects/8612326dfc09414caca0b216d53b9f13/users/ff1b16ee76d34b72b236b28dfbd94fb8/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:38.515726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324345-c1e5-46da-ac1e-832e268fbd92 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.537531 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27324345-c1e5-46da-ac1e-832e268fbd92 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.607941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:38.643532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8de68-60a5-4b20-964e-a01b70e73e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.644468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8de68-60a5-4b20-964e-a01b70e73e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.645764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8de68-60a5-4b20-964e-a01b70e73e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.646201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb8de68-60a5-4b20-964e-a01b70e73e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.647628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8de68-60a5-4b20-964e-a01b70e73e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.650512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb8de68-60a5-4b20-964e-a01b70e73e63 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.686281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376965d-9a1e-4e0a-afa4-05f8057136f0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:38.686738 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1309/2611] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:29:38.711701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376965d-9a1e-4e0a-afa4-05f8057136f0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:38.732782 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9d17fe-9342-45d5-927f-783ed1c7654f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:38.733783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d17fe-9342-45d5-927f-783ed1c7654f admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:38.734066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d17fe-9342-45d5-927f-783ed1c7654f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:38.734339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9d17fe-9342-45d5-927f-783ed1c7654f admin admin] PATH_INFO: `/v3/projects/8612326dfc09414caca0b216d53b9f13/users/ff1b16ee76d34b72b236b28dfbd94fb8/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:38.744155 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d17fe-9342-45d5-927f-783ed1c7654f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8612326dfc09414caca0b216d53b9f13, user_id=ff1b16ee76d34b72b236b28dfbd94fb8, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:38.746933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9d17fe-9342-45d5-927f-783ed1c7654f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:38.779995 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1303/2612] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:29:38 2022] PUT /identity/v3/projects/8612326dfc09414caca0b216d53b9f13/users/ff1b16ee76d34b72b236b28dfbd94fb8/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:42.483628 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8de68-60a5-4b20-964e-a01b70e73e63 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:42.629018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb8de68-60a5-4b20-964e-a01b70e73e63 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:42.796652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:42.798989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:42.805340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:42.806266 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:42.813000 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:43.134327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:43.144188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:43.186475 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1310/2613] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 704 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:47.671071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e29f3-c087-4222-a286-112ac52e25b8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:47.671619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e29f3-c087-4222-a286-112ac52e25b8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:47.671822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e29f3-c087-4222-a286-112ac52e25b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:47.719408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:47.731374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3f0e29f3-c087-4222-a286-112ac52e25b8 None None] MFA Rules not processed for user `ff1b16ee76d34b72b236b28dfbd94fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:47.732988 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3f0e29f3-c087-4222-a286-112ac52e25b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:47.850584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1304/2614] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:47 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:47.868267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:47.870464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:47.972221 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:48.033951 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca42e3a4-470e-4045-9ecc-f65184d1dbe1 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:48.044742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acba8cb-5cdb-458f-9031-b98d9ca5ac13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:48.046240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acba8cb-5cdb-458f-9031-b98d9ca5ac13 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:48.047188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acba8cb-5cdb-458f-9031-b98d9ca5ac13 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:48.047650 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acba8cb-5cdb-458f-9031-b98d9ca5ac13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:48.187109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acba8cb-5cdb-458f-9031-b98d9ca5ac13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:48.188673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acba8cb-5cdb-458f-9031-b98d9ca5ac13 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:48.202508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1311/2615] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:49.438870 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20013f-4ac2-41e1-a532-486100b15a5d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.440329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20013f-4ac2-41e1-a532-486100b15a5d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.441816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20013f-4ac2-41e1-a532-486100b15a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.495447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6e20013f-4ac2-41e1-a532-486100b15a5d None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:49.497826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6e20013f-4ac2-41e1-a532-486100b15a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:49.688557 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1305/2616] 10.222.0.66 () {58 vars in 967 bytes} [Thu Jun 16 17:29:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:49.743355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acba8cb-5cdb-458f-9031-b98d9ca5ac13 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:49.917368 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acba8cb-5cdb-458f-9031-b98d9ca5ac13 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:49.940940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:49.941989 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:49.943008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:49.943428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:49.969745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:49.977237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:50.018413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1312/2617] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:29:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:29:51.195242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732e76b-4f1d-4c41-8ec7-6068c178e0e2 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.195242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732e76b-4f1d-4c41-8ec7-6068c178e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.195993 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732e76b-4f1d-4c41-8ec7-6068c178e0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.239687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5732e76b-4f1d-4c41-8ec7-6068c178e0e2 None None] MFA Rules not processed for user `6cd1d0ce941845c3a28e2cd5fc77c844`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:51.241069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5732e76b-4f1d-4c41-8ec7-6068c178e0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:51.270872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1306/2618] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:51.598635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:51.600496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:51.728785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732e76b-4f1d-4c41-8ec7-6068c178e0e2 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:51.732397 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5732e76b-4f1d-4c41-8ec7-6068c178e0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:51.773773 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732e76b-4f1d-4c41-8ec7-6068c178e0e2 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:51.776965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb5bf4-f946-4cca-b7ff-ca3d9a9d3bec service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:51.780187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:51.781383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.781600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.782097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] PATH_INFO: `/v3/users/1a3418bd5d7f430ca84efd5c83e38fed` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.783476 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a3418bd5d7f430ca84efd5c83e38fed)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:51.787016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:51.790340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:51.791708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:51.793532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:51.794489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:51.799329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:51.836351 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:51.843797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:51.863915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1313/2619] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:51.906938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a3418bd5d7f430ca84efd5c83e38fed'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:51.933355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a3418bd5d7f430ca84efd5c83e38fed'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:51.934175 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a3418bd5d7f430ca84efd5c83e38fed'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:51.948489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a3418bd5d7f430ca84efd5c83e38fed'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:51.959955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-7bc00947-d7a3-4c91-ac86-ae479bef96d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a3418bd5d7f430ca84efd5c83e38fed'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:51.968467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1307/2620] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:29:51 2022] DELETE /identity/v3/users/1a3418bd5d7f430ca84efd5c83e38fed => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:52.293315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.307874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:52.319048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1e81f-699e-49cf-94a4-f35d2bfdef2c service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.403735 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.405601 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.406479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.407053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] PATH_INFO: `/v3/projects/f029a8bb389c4d348c0e9001c5fa059c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.413759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f029a8bb389c4d348c0e9001c5fa059c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.417545 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:52.487437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f029a8bb389c4d348c0e9001c5fa059c', 'request_id': 'req-d11e691a-567a-4902-9085-eea3a9eb5e26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:52.496488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f029a8bb389c4d348c0e9001c5fa059c', 'request_id': 'req-d11e691a-567a-4902-9085-eea3a9eb5e26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:52.506487 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f029a8bb389c4d348c0e9001c5fa059c', 'request_id': 'req-d11e691a-567a-4902-9085-eea3a9eb5e26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:52.527990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] The token cache is being invalidate because project f029a8bb389c4d348c0e9001c5fa059c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:29:52.528474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:52.529496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d11e691a-567a-4902-9085-eea3a9eb5e26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:29:52.532365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1314/2621] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:29:52 2022] DELETE /identity/v3/projects/f029a8bb389c4d348c0e9001c5fa059c => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:52.539467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26329523-d1ef-47d2-88ae-c6b2954104e8 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.540328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26329523-d1ef-47d2-88ae-c6b2954104e8 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.540493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26329523-d1ef-47d2-88ae-c6b2954104e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.564902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-26329523-d1ef-47d2-88ae-c6b2954104e8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:52.566286 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-26329523-d1ef-47d2-88ae-c6b2954104e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:52.573887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:52.651947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1308/2622] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:52.660623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fd329-25ca-4916-9726-0879c96e23a5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.661357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fd329-25ca-4916-9726-0879c96e23a5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.661357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5fd329-25ca-4916-9726-0879c96e23a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.679999 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ea5fd329-25ca-4916-9726-0879c96e23a5 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:52.681260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ea5fd329-25ca-4916-9726-0879c96e23a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:52.691092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1315/2623] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:52.695527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26329523-d1ef-47d2-88ae-c6b2954104e8 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.707055 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26329523-d1ef-47d2-88ae-c6b2954104e8 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.712599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.713540 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.713814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.714104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.714836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:29:52.715203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.716823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:52.730923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1309/2624] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:29:52.737205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fd329-25ca-4916-9726-0879c96e23a5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.747146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5fd329-25ca-4916-9726-0879c96e23a5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.752588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe47d8-bded-4879-a3a9-cd962fff2742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.752884 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe47d8-bded-4879-a3a9-cd962fff2742 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.753045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe47d8-bded-4879-a3a9-cd962fff2742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.753338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fe47d8-bded-4879-a3a9-cd962fff2742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.754040 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe47d8-bded-4879-a3a9-cd962fff2742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.756324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fe47d8-bded-4879-a3a9-cd962fff2742 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:52.805258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1316/2625] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:29:52 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:52.807393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.818470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c32eb9-17f1-4f85-904a-1c06ab5a10f9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.910016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e3e1e-8266-4b6b-a92e-90ba05b90516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.916381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e3e1e-8266-4b6b-a92e-90ba05b90516 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.916381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e3e1e-8266-4b6b-a92e-90ba05b90516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.916381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e3e1e-8266-4b6b-a92e-90ba05b90516 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.916381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e3e1e-8266-4b6b-a92e-90ba05b90516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.916381 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e3e1e-8266-4b6b-a92e-90ba05b90516 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:52.953233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1310/2626] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:29:52 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:29:52.959474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe47d8-bded-4879-a3a9-cd962fff2742 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:52.978986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fe47d8-bded-4879-a3a9-cd962fff2742 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:52.988554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368680a-6fb8-4988-a970-680f94090e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:52.989584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368680a-6fb8-4988-a970-680f94090e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:52.989936 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368680a-6fb8-4988-a970-680f94090e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:52.990563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368680a-6fb8-4988-a970-680f94090e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:52.991978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368680a-6fb8-4988-a970-680f94090e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:52.995107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368680a-6fb8-4988-a970-680f94090e4b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.012112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1317/2627] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:29:53.016232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e3e1e-8266-4b6b-a92e-90ba05b90516 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.028826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e3e1e-8266-4b6b-a92e-90ba05b90516 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.035746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9431e48c-2811-4eeb-a12f-eeed71beb634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.036667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431e48c-2811-4eeb-a12f-eeed71beb634 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.037051 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431e48c-2811-4eeb-a12f-eeed71beb634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.037446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431e48c-2811-4eeb-a12f-eeed71beb634 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.038340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431e48c-2811-4eeb-a12f-eeed71beb634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.040404 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9431e48c-2811-4eeb-a12f-eeed71beb634 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.064752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1311/2628] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:29:53.068396 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368680a-6fb8-4988-a970-680f94090e4b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.086753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368680a-6fb8-4988-a970-680f94090e4b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.097033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdd695-ad48-4906-b3e9-41690e57f49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.098026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdd695-ad48-4906-b3e9-41690e57f49f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.098436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdd695-ad48-4906-b3e9-41690e57f49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.098740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fdd695-ad48-4906-b3e9-41690e57f49f admin admin] PATH_INFO: `/v3/projects/0665ded71fed48f0abc6da49edee2e26/users/7e0c77d61627415db92b299ee566f427/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.122604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdd695-ad48-4906-b3e9-41690e57f49f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0665ded71fed48f0abc6da49edee2e26, user_id=7e0c77d61627415db92b299ee566f427, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.125607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fdd695-ad48-4906-b3e9-41690e57f49f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.146614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1318/2629] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:29:53 2022] PUT /identity/v3/projects/0665ded71fed48f0abc6da49edee2e26/users/7e0c77d61627415db92b299ee566f427/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:53.157823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70624aab-f12a-4a6e-bedd-e5e10cb1633c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.158158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70624aab-f12a-4a6e-bedd-e5e10cb1633c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.158394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70624aab-f12a-4a6e-bedd-e5e10cb1633c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.203098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-70624aab-f12a-4a6e-bedd-e5e10cb1633c None None] MFA Rules not processed for user `7e0c77d61627415db92b299ee566f427`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:53.206359 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-70624aab-f12a-4a6e-bedd-e5e10cb1633c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:53.299421 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1312/2630] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:29:53.304200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdd695-ad48-4906-b3e9-41690e57f49f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.352110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.405483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fdd695-ad48-4906-b3e9-41690e57f49f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.410794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faad2d6-da21-4a36-967c-688d10f758a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.411488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faad2d6-da21-4a36-967c-688d10f758a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.411777 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faad2d6-da21-4a36-967c-688d10f758a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.412252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faad2d6-da21-4a36-967c-688d10f758a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.413874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faad2d6-da21-4a36-967c-688d10f758a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.416069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faad2d6-da21-4a36-967c-688d10f758a5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.459904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1319/2631] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:29:53 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:29:53.462254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70624aab-f12a-4a6e-bedd-e5e10cb1633c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.472309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:53.480537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70624aab-f12a-4a6e-bedd-e5e10cb1633c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.563799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9588-0ee9-4023-8814-d0c86c06f629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.564640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9588-0ee9-4023-8814-d0c86c06f629 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.564964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9588-0ee9-4023-8814-d0c86c06f629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.565272 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f9588-0ee9-4023-8814-d0c86c06f629 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.566190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f9588-0ee9-4023-8814-d0c86c06f629 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.568305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f9588-0ee9-4023-8814-d0c86c06f629 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.597721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1313/2632] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:29:53 2022] POST /identity/v3/users => generated 292 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:29:53.599380 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faad2d6-da21-4a36-967c-688d10f758a5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.612881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faad2d6-da21-4a36-967c-688d10f758a5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.618594 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc78c3-3837-4a6c-b879-c122166ffd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.619346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc78c3-3837-4a6c-b879-c122166ffd69 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.619645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc78c3-3837-4a6c-b879-c122166ffd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.619986 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc78c3-3837-4a6c-b879-c122166ffd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.620743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc78c3-3837-4a6c-b879-c122166ffd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.622391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc78c3-3837-4a6c-b879-c122166ffd69 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.635541 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1320/2633] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:29:53.640234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9588-0ee9-4023-8814-d0c86c06f629 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.658880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f9588-0ee9-4023-8814-d0c86c06f629 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.667778 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210aae-eeb8-44cb-8469-7f118a80ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.668547 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210aae-eeb8-44cb-8469-7f118a80ffd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.668825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210aae-eeb8-44cb-8469-7f118a80ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.669095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6210aae-eeb8-44cb-8469-7f118a80ffd5 admin admin] PATH_INFO: `/v3/projects/d95e3f3448e54291bbeb1dddd1349efa/users/7f216823e5394337ba19fdeb5c61c9ae/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.689348 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210aae-eeb8-44cb-8469-7f118a80ffd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95e3f3448e54291bbeb1dddd1349efa, user_id=7f216823e5394337ba19fdeb5c61c9ae, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.692424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6210aae-eeb8-44cb-8469-7f118a80ffd5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.711084 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1314/2634] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:29:53 2022] PUT /identity/v3/projects/d95e3f3448e54291bbeb1dddd1349efa/users/7f216823e5394337ba19fdeb5c61c9ae/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:53.715070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc78c3-3837-4a6c-b879-c122166ffd69 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.725965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc78c3-3837-4a6c-b879-c122166ffd69 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.808719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc94d5d-5fab-4c7b-8819-af826264bce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.809500 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc94d5d-5fab-4c7b-8819-af826264bce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.810143 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc94d5d-5fab-4c7b-8819-af826264bce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.810418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc94d5d-5fab-4c7b-8819-af826264bce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.811506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc94d5d-5fab-4c7b-8819-af826264bce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.813392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc94d5d-5fab-4c7b-8819-af826264bce4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.825907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1321/2635] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:29:53.830555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210aae-eeb8-44cb-8469-7f118a80ffd5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:53.843153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6210aae-eeb8-44cb-8469-7f118a80ffd5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:53.848109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:53.849269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.849555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.849828 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] PATH_INFO: `/v3/projects/d95e3f3448e54291bbeb1dddd1349efa/users/7f216823e5394337ba19fdeb5c61c9ae/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.852052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95e3f3448e54291bbeb1dddd1349efa, user_id=7f216823e5394337ba19fdeb5c61c9ae, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:53.854744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:53.872091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1315/2636] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:29:53 2022] PUT /identity/v3/projects/d95e3f3448e54291bbeb1dddd1349efa/users/7f216823e5394337ba19fdeb5c61c9ae/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:29:53.883204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03150e-ab1f-4032-8c05-abb44565401b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:53.883665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03150e-ab1f-4032-8c05-abb44565401b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:53.884108 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03150e-ab1f-4032-8c05-abb44565401b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:53.928737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3f03150e-ab1f-4032-8c05-abb44565401b None None] MFA Rules not processed for user `7f216823e5394337ba19fdeb5c61c9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:29:53.928737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f03150e-ab1f-4032-8c05-abb44565401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:54.029840 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1322/2637] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:29:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:29:54.039576 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:54.040860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:54.106761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3017dff-de3c-4699-b937-5df1cc7c3a83 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:54.111049 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:54.112052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:54.112337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:54.112615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:54.123982 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:54.125637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:54.139374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1316/2638] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:29:54.268990 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03150e-ab1f-4032-8c05-abb44565401b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:29:54.269301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f03150e-ab1f-4032-8c05-abb44565401b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:29:54.284558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03150e-ab1f-4032-8c05-abb44565401b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:29:54.291357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94668f-0089-434d-acdb-04b7efd7bc5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:29:54.294233 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94668f-0089-434d-acdb-04b7efd7bc5a service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:29:54.294674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94668f-0089-434d-acdb-04b7efd7bc5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:29:54.295113 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94668f-0089-434d-acdb-04b7efd7bc5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:29:54.330900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:29:54.376659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94668f-0089-434d-acdb-04b7efd7bc5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:29:54.378161 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94668f-0089-434d-acdb-04b7efd7bc5a service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:29:54.390389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1323/2639] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:29:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:04.133015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:04.385700 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=17 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:26.545479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:26.547160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:26.616297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057c6e-6c8b-41e3-951e-522472e274d0 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:26.619877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0afa99-2297-4a4b-b4ba-2fe9188efa6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:26.620525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0afa99-2297-4a4b-b4ba-2fe9188efa6b service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:26.620802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0afa99-2297-4a4b-b4ba-2fe9188efa6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:26.621074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0afa99-2297-4a4b-b4ba-2fe9188efa6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:26.623012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5d0afa99-2297-4a4b-b4ba-2fe9188efa6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:26.696424 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0afa99-2297-4a4b-b4ba-2fe9188efa6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:26.697915 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0afa99-2297-4a4b-b4ba-2fe9188efa6b service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:26.708961 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1317/2640] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:35.046285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94668f-0089-434d-acdb-04b7efd7bc5a service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:35.123880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94668f-0089-434d-acdb-04b7efd7bc5a service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:35.130615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:35.131526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:35.131815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:35.132092 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] PATH_INFO: `/v3/users/7e0c77d61627415db92b299ee566f427` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:35.133444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0c77d61627415db92b299ee566f427)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:35.135426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:35.200407 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0c77d61627415db92b299ee566f427'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.201097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0c77d61627415db92b299ee566f427'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.201748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0c77d61627415db92b299ee566f427'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.222665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0c77d61627415db92b299ee566f427'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.233794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0c77d61627415db92b299ee566f427'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.253104 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1324/2641] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:30:35 2022] DELETE /identity/v3/users/7e0c77d61627415db92b299ee566f427 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:35.257948 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf218e7-16d1-49a9-8e18-9b69b1319122 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:35.258552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf218e7-16d1-49a9-8e18-9b69b1319122 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:35.258552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf218e7-16d1-49a9-8e18-9b69b1319122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:35.283927 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-acf218e7-16d1-49a9-8e18-9b69b1319122 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:35.285074 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-acf218e7-16d1-49a9-8e18-9b69b1319122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:35.351232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1318/2642] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:35.362659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:35.365102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:35.431075 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73bba6-6a7b-4359-9098-ef012bdb26b9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:35.436752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc5558-16a7-402c-ad7d-c250aa5bd791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:35.437617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5558-16a7-402c-ad7d-c250aa5bd791 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:35.437907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5558-16a7-402c-ad7d-c250aa5bd791 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:35.438186 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5558-16a7-402c-ad7d-c250aa5bd791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:35.451726 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc5558-16a7-402c-ad7d-c250aa5bd791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:35.453282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc5558-16a7-402c-ad7d-c250aa5bd791 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:35.465767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1325/2643] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:35.770091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf218e7-16d1-49a9-8e18-9b69b1319122 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:35.771817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-acf218e7-16d1-49a9-8e18-9b69b1319122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:35.786275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf218e7-16d1-49a9-8e18-9b69b1319122 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:35.798415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:35.799415 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:35.799768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:35.800174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] PATH_INFO: `/v3/projects/0665ded71fed48f0abc6da49edee2e26` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:35.801856 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0665ded71fed48f0abc6da49edee2e26)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:35.803796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:35.868587 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:35.871206 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0665ded71fed48f0abc6da49edee2e26', 'request_id': 'req-15658cf4-6df2-45f5-9c30-3be7480e5355'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.879796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0665ded71fed48f0abc6da49edee2e26', 'request_id': 'req-15658cf4-6df2-45f5-9c30-3be7480e5355'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.886383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0665ded71fed48f0abc6da49edee2e26', 'request_id': 'req-15658cf4-6df2-45f5-9c30-3be7480e5355'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.899570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] The token cache is being invalidate because project 0665ded71fed48f0abc6da49edee2e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:30:35.899860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.900589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:35.902227 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1319/2644] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:30:35 2022] DELETE /identity/v3/projects/0665ded71fed48f0abc6da49edee2e26 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:35.906350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc5558-16a7-402c-ad7d-c250aa5bd791 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:35.916773 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc5558-16a7-402c-ad7d-c250aa5bd791 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:35.939390 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:35.984042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:35.985337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:35.985624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:35.985888 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] PATH_INFO: `/v3/users/7f216823e5394337ba19fdeb5c61c9ae` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:35.987491 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f216823e5394337ba19fdeb5c61c9ae)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:35.989578 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:36.045496 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f216823e5394337ba19fdeb5c61c9ae'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.046214 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f216823e5394337ba19fdeb5c61c9ae'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.046787 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f216823e5394337ba19fdeb5c61c9ae'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.059707 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f216823e5394337ba19fdeb5c61c9ae'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.068723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2cf4c1eb-2b09-493f-a15a-a70b2a0e0b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f216823e5394337ba19fdeb5c61c9ae'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.083621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1326/2645] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:30:35 2022] DELETE /identity/v3/users/7f216823e5394337ba19fdeb5c61c9ae => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:36.415743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:36.417408 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:36.459017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15658cf4-6df2-45f5-9c30-3be7480e5355 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:36.546317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:36.547340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.547687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.548213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] PATH_INFO: `/v3/projects/d95e3f3448e54291bbeb1dddd1349efa` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.549873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95e3f3448e54291bbeb1dddd1349efa)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:36.552082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:36.620125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95e3f3448e54291bbeb1dddd1349efa', 'request_id': 'req-3358b010-47fc-4d97-b865-69ceee0a6086'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.630201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95e3f3448e54291bbeb1dddd1349efa', 'request_id': 'req-3358b010-47fc-4d97-b865-69ceee0a6086'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.646619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95e3f3448e54291bbeb1dddd1349efa', 'request_id': 'req-3358b010-47fc-4d97-b865-69ceee0a6086'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.658813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] The token cache is being invalidate because project d95e3f3448e54291bbeb1dddd1349efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:30:36.659160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.659843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3358b010-47fc-4d97-b865-69ceee0a6086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:36.662262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1320/2646] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:30:36 2022] DELETE /identity/v3/projects/d95e3f3448e54291bbeb1dddd1349efa => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:36.667646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851f0a-6a57-4482-9c45-b362cfed1027 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.667933 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851f0a-6a57-4482-9c45-b362cfed1027 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.668240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a851f0a-6a57-4482-9c45-b362cfed1027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.688530 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5a851f0a-6a57-4482-9c45-b362cfed1027 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:36.689783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a851f0a-6a57-4482-9c45-b362cfed1027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:36.771580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1327/2647] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:36.780580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0b3dc-efc0-4658-95b4-30d401fe88d5 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.780879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0b3dc-efc0-4658-95b4-30d401fe88d5 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.781157 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0b3dc-efc0-4658-95b4-30d401fe88d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.804280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-48f0b3dc-efc0-4658-95b4-30d401fe88d5 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:36.806158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-48f0b3dc-efc0-4658-95b4-30d401fe88d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:36.816565 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1321/2648] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:30:36.820564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a851f0a-6a57-4482-9c45-b362cfed1027 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:36.830606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a851f0a-6a57-4482-9c45-b362cfed1027 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:36.835756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:36.836568 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.837138 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.837550 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.839945 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:36.840403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:36.842434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:36.861579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1328/2649] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:36.865619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0b3dc-efc0-4658-95b4-30d401fe88d5 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:36.875688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0b3dc-efc0-4658-95b4-30d401fe88d5 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:36.880684 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76a756-6f84-4a82-a6af-9c1a2d97bac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:36.881420 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76a756-6f84-4a82-a6af-9c1a2d97bac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.881730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76a756-6f84-4a82-a6af-9c1a2d97bac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.882635 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76a756-6f84-4a82-a6af-9c1a2d97bac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.882759 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76a756-6f84-4a82-a6af-9c1a2d97bac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:36.884614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76a756-6f84-4a82-a6af-9c1a2d97bac0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:36.919112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1322/2650] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:30:36 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:36.920585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:36.931190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f04892-6f8d-4d4c-a375-1b0adea3cfb7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:36.959144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:36.993684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6452fa-19a4-4b46-8185-762279814e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:36.994696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6452fa-19a4-4b46-8185-762279814e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:36.995020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6452fa-19a4-4b46-8185-762279814e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:36.995322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6452fa-19a4-4b46-8185-762279814e11 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:36.996044 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6452fa-19a4-4b46-8185-762279814e11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:36.998146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6452fa-19a4-4b46-8185-762279814e11 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.027582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1329/2651] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:30:36 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:37.028786 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76a756-6f84-4a82-a6af-9c1a2d97bac0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.040444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76a756-6f84-4a82-a6af-9c1a2d97bac0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.045854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1d279-0be2-4f58-ad3e-97b304b85afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.046652 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1d279-0be2-4f58-ad3e-97b304b85afd admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.046939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1d279-0be2-4f58-ad3e-97b304b85afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.047217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1d279-0be2-4f58-ad3e-97b304b85afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.048162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1d279-0be2-4f58-ad3e-97b304b85afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:37.049838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1d279-0be2-4f58-ad3e-97b304b85afd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.064228 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1323/2652] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:37.065817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6452fa-19a4-4b46-8185-762279814e11 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.079374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6452fa-19a4-4b46-8185-762279814e11 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.088068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20971abb-9b4a-41d3-8f92-5fac2882ee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.088952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20971abb-9b4a-41d3-8f92-5fac2882ee2f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.089280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20971abb-9b4a-41d3-8f92-5fac2882ee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.089756 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20971abb-9b4a-41d3-8f92-5fac2882ee2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.090794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20971abb-9b4a-41d3-8f92-5fac2882ee2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:37.092434 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20971abb-9b4a-41d3-8f92-5fac2882ee2f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.104917 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1330/2653] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:37.109054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1d279-0be2-4f58-ad3e-97b304b85afd admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.121139 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1d279-0be2-4f58-ad3e-97b304b85afd admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.126253 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.127357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.127711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.128078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] PATH_INFO: `/v3/projects/f52cdb28f77d4fccb0f3dcad365626d4/users/4cbdfca9fb9c40fabbe16bb5541ef1d7/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.150315 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f52cdb28f77d4fccb0f3dcad365626d4, user_id=4cbdfca9fb9c40fabbe16bb5541ef1d7, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:37.153148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.168474 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:37.175291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1324/2654] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:30:37 2022] PUT /identity/v3/projects/f52cdb28f77d4fccb0f3dcad365626d4/users/4cbdfca9fb9c40fabbe16bb5541ef1d7/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:37.182899 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb159a-cc7a-4699-b624-6a31cd15c345 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.183297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb159a-cc7a-4699-b624-6a31cd15c345 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.183665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb159a-cc7a-4699-b624-6a31cd15c345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.218135 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6fbb159a-cc7a-4699-b624-6a31cd15c345 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:37.219471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6fbb159a-cc7a-4699-b624-6a31cd15c345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:37.298741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1331/2655] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:37.306048 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:37.307849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:37.394136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5f8de-1759-4c54-bd2d-e478e2f70cc9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:37.398328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5250f-c12f-4098-bba2-228a779991fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:37.399010 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5250f-c12f-4098-bba2-228a779991fe service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:37.399302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5250f-c12f-4098-bba2-228a779991fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:37.399579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5250f-c12f-4098-bba2-228a779991fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:37.412276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5250f-c12f-4098-bba2-228a779991fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:37.413798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f5250f-c12f-4098-bba2-228a779991fe service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:37.427130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1325/2656] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:38.180740 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda76806-d9fc-42ae-8ccb-17917caa74ad None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.181333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda76806-d9fc-42ae-8ccb-17917caa74ad None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.181755 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda76806-d9fc-42ae-8ccb-17917caa74ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.218580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cda76806-d9fc-42ae-8ccb-17917caa74ad None None] MFA Rules not processed for user `4cbdfca9fb9c40fabbe16bb5541ef1d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:38.220098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cda76806-d9fc-42ae-8ccb-17917caa74ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:38.310896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1332/2657] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:38.315152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5250f-c12f-4098-bba2-228a779991fe service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.326320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5250f-c12f-4098-bba2-228a779991fe service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.331582 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4af307-8b05-4953-91d2-39ddc910fac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.332414 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4af307-8b05-4953-91d2-39ddc910fac5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.332677 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4af307-8b05-4953-91d2-39ddc910fac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.332968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4af307-8b05-4953-91d2-39ddc910fac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.334518 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4af307-8b05-4953-91d2-39ddc910fac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.336380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4af307-8b05-4953-91d2-39ddc910fac5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.374788 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1326/2658] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:30:38 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:38.383537 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda76806-d9fc-42ae-8ccb-17917caa74ad None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.396120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda76806-d9fc-42ae-8ccb-17917caa74ad None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.450089 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:38.483857 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da91666-e901-482c-a187-70e434b58e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.484539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91666-e901-482c-a187-70e434b58e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.484824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91666-e901-482c-a187-70e434b58e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.485098 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da91666-e901-482c-a187-70e434b58e2c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.485831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da91666-e901-482c-a187-70e434b58e2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.488076 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da91666-e901-482c-a187-70e434b58e2c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.528034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1333/2659] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:30:38 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:38.532360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4af307-8b05-4953-91d2-39ddc910fac5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.543482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4af307-8b05-4953-91d2-39ddc910fac5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.549920 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a0a43-bff4-40df-9ec3-d1c434bf7cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.550907 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a0a43-bff4-40df-9ec3-d1c434bf7cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.551506 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a0a43-bff4-40df-9ec3-d1c434bf7cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.551946 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a0a43-bff4-40df-9ec3-d1c434bf7cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.552860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a0a43-bff4-40df-9ec3-d1c434bf7cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.555295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a0a43-bff4-40df-9ec3-d1c434bf7cb3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.573006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da91666-e901-482c-a187-70e434b58e2c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.573499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1327/2660] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:38.584513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da91666-e901-482c-a187-70e434b58e2c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.588886 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208dcfd-a7e2-47ec-9e9e-4576fb237c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.589955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208dcfd-a7e2-47ec-9e9e-4576fb237c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.590344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208dcfd-a7e2-47ec-9e9e-4576fb237c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.590729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208dcfd-a7e2-47ec-9e9e-4576fb237c02 admin admin] PATH_INFO: `/v3/projects/64deac84e2424b43af72167c90ee37cd/users/f8dfb817205243b2b45a07ca26f94ca7/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.608276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208dcfd-a7e2-47ec-9e9e-4576fb237c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64deac84e2424b43af72167c90ee37cd, user_id=f8dfb817205243b2b45a07ca26f94ca7, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.611145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2208dcfd-a7e2-47ec-9e9e-4576fb237c02 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.628835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1334/2661] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:30:38 2022] PUT /identity/v3/projects/64deac84e2424b43af72167c90ee37cd/users/f8dfb817205243b2b45a07ca26f94ca7/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:30:38.633402 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a0a43-bff4-40df-9ec3-d1c434bf7cb3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.643800 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a0a43-bff4-40df-9ec3-d1c434bf7cb3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.709337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef70a9-8bd9-4e44-bb52-ad2c37096828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.710081 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef70a9-8bd9-4e44-bb52-ad2c37096828 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.710400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef70a9-8bd9-4e44-bb52-ad2c37096828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.710645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef70a9-8bd9-4e44-bb52-ad2c37096828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.711427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef70a9-8bd9-4e44-bb52-ad2c37096828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.713016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef70a9-8bd9-4e44-bb52-ad2c37096828 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.727204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1328/2662] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:38.728625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208dcfd-a7e2-47ec-9e9e-4576fb237c02 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:38.740911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2208dcfd-a7e2-47ec-9e9e-4576fb237c02 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:38.746121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:38.747039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:38.747304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:38.747606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] PATH_INFO: `/v3/projects/64deac84e2424b43af72167c90ee37cd/users/f8dfb817205243b2b45a07ca26f94ca7/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:38.749732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64deac84e2424b43af72167c90ee37cd, user_id=f8dfb817205243b2b45a07ca26f94ca7, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:38.752405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:38.764403 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1335/2663] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:30:38 2022] PUT /identity/v3/projects/64deac84e2424b43af72167c90ee37cd/users/f8dfb817205243b2b45a07ca26f94ca7/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:30:40.039303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76beace-1a0f-47fb-b523-534cb59cdaeb None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.039855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76beace-1a0f-47fb-b523-534cb59cdaeb None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.039941 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76beace-1a0f-47fb-b523-534cb59cdaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.079609 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-e76beace-1a0f-47fb-b523-534cb59cdaeb None None] MFA Rules not processed for user `f8dfb817205243b2b45a07ca26f94ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:40.080852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-e76beace-1a0f-47fb-b523-534cb59cdaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:40.178606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:40.216793 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1329/2664] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:40.243501 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:40.246352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:40.363231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6658e6b-e050-4633-9675-ea5487b87033 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:40.370395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5138c7-f346-41fe-936c-341ad9e0a5f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:40.371375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5138c7-f346-41fe-936c-341ad9e0a5f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:40.371812 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5138c7-f346-41fe-936c-341ad9e0a5f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:40.372107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5138c7-f346-41fe-936c-341ad9e0a5f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:40.446185 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5138c7-f346-41fe-936c-341ad9e0a5f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:40.447734 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5138c7-f346-41fe-936c-341ad9e0a5f3 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:40.458519 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1336/2665] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:41.854664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76beace-1a0f-47fb-b523-534cb59cdaeb None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:41.868836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76beace-1a0f-47fb-b523-534cb59cdaeb None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:41.876642 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:41.877394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:41.877675 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:41.877933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:41.893608 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:41.896464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:41.910070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1330/2666] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:41 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:42.671426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5138c7-f346-41fe-936c-341ad9e0a5f3 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:42.724019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:42.733881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5138c7-f346-41fe-936c-341ad9e0a5f3 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:42.738975 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:42.739227 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:42.739358 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:42.739634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:42.741564 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:42.818563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:42.820930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:42.832386 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1337/2667] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:46.538142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.539682 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:46.555696 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5fe6fb-e467-4f00-af7d-d2ad8c7d4301 service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.561245 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.562233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.562600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.563066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] PATH_INFO: `/v3/users/68da2fd51cc84b2b8d641e1e6ac17c72` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.564457 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68da2fd51cc84b2b8d641e1e6ac17c72)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:46.566890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:46.624034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68da2fd51cc84b2b8d641e1e6ac17c72'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:46.637207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68da2fd51cc84b2b8d641e1e6ac17c72'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:46.637965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68da2fd51cc84b2b8d641e1e6ac17c72'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:46.647872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68da2fd51cc84b2b8d641e1e6ac17c72'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:46.656486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-aa7c312c-db3b-4670-acc3-3a9921a57309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68da2fd51cc84b2b8d641e1e6ac17c72'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:46.663747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1331/2668] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:30:46 2022] DELETE /identity/v3/users/68da2fd51cc84b2b8d641e1e6ac17c72 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:46.872164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:46.887424 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabb21e-126e-4254-922b-1bcfc804ea4a service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:46.959476 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:46.960362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:46.960648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:46.960921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] PATH_INFO: `/v3/projects/0c63f648631a4a11910b4b135f22b235` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:46.962181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c63f648631a4a11910b4b135f22b235)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:46.964239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.017703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c63f648631a4a11910b4b135f22b235', 'request_id': 'req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:47.027719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c63f648631a4a11910b4b135f22b235', 'request_id': 'req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:47.038133 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c63f648631a4a11910b4b135f22b235', 'request_id': 'req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:47.051421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] The token cache is being invalidate because project 0c63f648631a4a11910b4b135f22b235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:30:47.051714 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:47.054369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5eb3ade2-b13b-4450-9d9c-73f78d3f82da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:47.054369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1338/2669] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:30:46 2022] DELETE /identity/v3/projects/0c63f648631a4a11910b4b135f22b235 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:47.059559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8af5e-a495-487e-9473-528a37838ad6 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.059849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8af5e-a495-487e-9473-528a37838ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.060149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8af5e-a495-487e-9473-528a37838ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.087007 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-46e8af5e-a495-487e-9473-528a37838ad6 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:47.088632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-46e8af5e-a495-487e-9473-528a37838ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:47.167161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1332/2670] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:47.176010 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6166027-94a4-4d70-bc00-1fccfa5efb97 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.176350 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6166027-94a4-4d70-bc00-1fccfa5efb97 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.176641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6166027-94a4-4d70-bc00-1fccfa5efb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.197265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c6166027-94a4-4d70-bc00-1fccfa5efb97 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:47.198460 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c6166027-94a4-4d70-bc00-1fccfa5efb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:47.209129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1339/2671] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:47.216266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8af5e-a495-487e-9473-528a37838ad6 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.227667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8af5e-a495-487e-9473-528a37838ad6 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.232556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.236109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.236109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.236109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.236109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:47.236109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.236525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.249503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1333/2672] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:47.253966 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6166027-94a4-4d70-bc00-1fccfa5efb97 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.263916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6166027-94a4-4d70-bc00-1fccfa5efb97 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.269526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828856af-efa5-4284-835a-4f5b07a8b75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.270313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828856af-efa5-4284-835a-4f5b07a8b75c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.270599 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828856af-efa5-4284-835a-4f5b07a8b75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.270875 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828856af-efa5-4284-835a-4f5b07a8b75c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.271581 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828856af-efa5-4284-835a-4f5b07a8b75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.273570 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828856af-efa5-4284-835a-4f5b07a8b75c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.288859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:47.306095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1340/2673] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:30:47 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:47.307247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.317904 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295716ec-9851-4124-9fe0-32d96310b51b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.355651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:47.386119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ab9a-23bb-444c-95b1-099dbb49ed42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.386973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ab9a-23bb-444c-95b1-099dbb49ed42 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.387357 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ab9a-23bb-444c-95b1-099dbb49ed42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.387764 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ab9a-23bb-444c-95b1-099dbb49ed42 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.388615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6ab9a-23bb-444c-95b1-099dbb49ed42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.390671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6ab9a-23bb-444c-95b1-099dbb49ed42 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.415801 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1334/2674] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:30:47 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:47.417099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828856af-efa5-4284-835a-4f5b07a8b75c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.429606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828856af-efa5-4284-835a-4f5b07a8b75c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.434879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe449553-2c49-432d-a2e3-98089ff1d477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.435607 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449553-2c49-432d-a2e3-98089ff1d477 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.435898 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449553-2c49-432d-a2e3-98089ff1d477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.436178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449553-2c49-432d-a2e3-98089ff1d477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.436916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe449553-2c49-432d-a2e3-98089ff1d477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.438556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe449553-2c49-432d-a2e3-98089ff1d477 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.452109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1341/2675] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:30:47.456180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ab9a-23bb-444c-95b1-099dbb49ed42 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.467648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ab9a-23bb-444c-95b1-099dbb49ed42 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.472364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3dcfb0-35d5-4086-936e-4af2325ddaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.473814 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dcfb0-35d5-4086-936e-4af2325ddaaf admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.474172 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dcfb0-35d5-4086-936e-4af2325ddaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.474453 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3dcfb0-35d5-4086-936e-4af2325ddaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.475207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3dcfb0-35d5-4086-936e-4af2325ddaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.476931 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3dcfb0-35d5-4086-936e-4af2325ddaaf admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.488266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1335/2676] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:47.493145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe449553-2c49-432d-a2e3-98089ff1d477 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.508642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe449553-2c49-432d-a2e3-98089ff1d477 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.516780 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.517729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.518179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.518505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] PATH_INFO: `/v3/projects/7c56d4c430c44df49f48fe0b49ebcbd2/users/642ef5f213254d818de4888cca274311/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.537641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c56d4c430c44df49f48fe0b49ebcbd2, user_id=642ef5f213254d818de4888cca274311, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.540646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.562413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1342/2677] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:30:47 2022] PUT /identity/v3/projects/7c56d4c430c44df49f48fe0b49ebcbd2/users/642ef5f213254d818de4888cca274311/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:47.568726 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed7d3e-3e72-47ac-872c-498d5e554c86 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.569229 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed7d3e-3e72-47ac-872c-498d5e554c86 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.569821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed7d3e-3e72-47ac-872c-498d5e554c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.589562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-a1ed7d3e-3e72-47ac-872c-498d5e554c86 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:47.591313 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a1ed7d3e-3e72-47ac-872c-498d5e554c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:47.681012 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:47.681669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:47.684148 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1336/2678] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:47.754982 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b3e53-93ee-4a6d-9568-3ea3cacc5f77 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:47.758872 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:47.759653 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:47.759942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:47.760216 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:47.772665 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:47.774172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:47.786827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1343/2679] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:30:52.297341 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19f60f-251a-4164-a419-baea4399f2ab None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.298137 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19f60f-251a-4164-a419-baea4399f2ab None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.298439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19f60f-251a-4164-a419-baea4399f2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.330947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-9d19f60f-251a-4164-a419-baea4399f2ab None None] MFA Rules not processed for user `642ef5f213254d818de4888cca274311`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:52.333328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9d19f60f-251a-4164-a419-baea4399f2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:52.419658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1337/2680] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:52.439286 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:52.440817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:52.555543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e933f1-4b3f-4870-ab8a-2c47b5b47284 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:52.563365 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:52.564095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:52.564399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:52.564736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:52.575329 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:52.576959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:52.587847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1344/2681] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:30:54.205978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350003-1ca6-4b65-9174-05a161fd6e3d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:54.206520 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350003-1ca6-4b65-9174-05a161fd6e3d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:54.206520 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b350003-1ca6-4b65-9174-05a161fd6e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:54.225688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-4b350003-1ca6-4b65-9174-05a161fd6e3d None None] MFA Rules not processed for user `642ef5f213254d818de4888cca274311`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:54.226985 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4b350003-1ca6-4b65-9174-05a161fd6e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:54.234305 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1338/2682] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:55.380896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.401621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:55.559339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:55.582356 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b350003-1ca6-4b65-9174-05a161fd6e3d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:55.583959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-4b350003-1ca6-4b65-9174-05a161fd6e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:55.590801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fff60-4cb9-44c8-b3ac-29aeed3f8723 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.601610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.602398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.602650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b350003-1ca6-4b65-9174-05a161fd6e3d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:55.602782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.602950 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.605558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:55.609742 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:55.610552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:55.611119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:55.611643 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:55.614280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:55.625937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.627940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.634926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:55.637410 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:55.640748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1345/2683] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:55.679197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1339/2684] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:30:56.611691 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:56.613940 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:56.638506 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80844ab-ea22-4a63-9025-70d7455f94d2 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:56.644208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:56.645724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:56.646178 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:56.646648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] PATH_INFO: `/v3/users/4cbdfca9fb9c40fabbe16bb5541ef1d7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:56.648223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cbdfca9fb9c40fabbe16bb5541ef1d7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:56.651337 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:56.731775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cbdfca9fb9c40fabbe16bb5541ef1d7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:56.732481 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cbdfca9fb9c40fabbe16bb5541ef1d7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:56.735697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cbdfca9fb9c40fabbe16bb5541ef1d7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:56.749838 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cbdfca9fb9c40fabbe16bb5541ef1d7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:56.758535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cbdfca9fb9c40fabbe16bb5541ef1d7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:56.777225 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1346/2685] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:30:56 2022] DELETE /identity/v3/users/4cbdfca9fb9c40fabbe16bb5541ef1d7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:57.119489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.136689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9183acf-0b22-46c8-9ffc-55069ea2dde2 service placement] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.189791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.248967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.251079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.251499 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.251818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] PATH_INFO: `/v3/projects/f52cdb28f77d4fccb0f3dcad365626d4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.254195 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f52cdb28f77d4fccb0f3dcad365626d4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.258180 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.340523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f52cdb28f77d4fccb0f3dcad365626d4', 'request_id': 'req-ff3ce175-0433-4743-9593-30ec4171fea0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.353473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f52cdb28f77d4fccb0f3dcad365626d4', 'request_id': 'req-ff3ce175-0433-4743-9593-30ec4171fea0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.366044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f52cdb28f77d4fccb0f3dcad365626d4', 'request_id': 'req-ff3ce175-0433-4743-9593-30ec4171fea0'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.384181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] The token cache is being invalidate because project f52cdb28f77d4fccb0f3dcad365626d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:30:57.384584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.385386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.387107 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1340/2686] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:30:57 2022] DELETE /identity/v3/projects/f52cdb28f77d4fccb0f3dcad365626d4 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:57.391209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.392705 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:57.406405 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15605524-d18a-4339-a83e-1106ac173180 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.466643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.467544 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.467831 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.468110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] PATH_INFO: `/v3/users/f8dfb817205243b2b45a07ca26f94ca7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.469683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8dfb817205243b2b45a07ca26f94ca7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.472267 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.533762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8dfb817205243b2b45a07ca26f94ca7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.535062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8dfb817205243b2b45a07ca26f94ca7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.536879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8dfb817205243b2b45a07ca26f94ca7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.543911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:57.559543 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8dfb817205243b2b45a07ca26f94ca7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.569291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3707c238-27fb-4ccf-9c7c-be2d7074774e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8dfb817205243b2b45a07ca26f94ca7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:57.585806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1347/2687] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:30:57 2022] DELETE /identity/v3/users/f8dfb817205243b2b45a07ca26f94ca7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:57.818217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:57.819924 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:57.840342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ce175-0433-4743-9593-30ec4171fea0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:57.927751 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:57.929418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:57.929418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:57.929418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] PATH_INFO: `/v3/projects/64deac84e2424b43af72167c90ee37cd` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:57.930843 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64deac84e2424b43af72167c90ee37cd)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:57.932709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:57.991673 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64deac84e2424b43af72167c90ee37cd', 'request_id': 'req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:58.002746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64deac84e2424b43af72167c90ee37cd', 'request_id': 'req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:58.014351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.015461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64deac84e2424b43af72167c90ee37cd', 'request_id': 'req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:58.031771 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] The token cache is being invalidate because project 64deac84e2424b43af72167c90ee37cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:30:58.032077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:58.033151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-0f8b2915-e6a4-430c-97e8-83b6a11c8c02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:30:58.034784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1341/2688] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:30:57 2022] DELETE /identity/v3/projects/64deac84e2424b43af72167c90ee37cd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:58.040320 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42714ac1-a579-4562-9b4d-ed835896273f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.040905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42714ac1-a579-4562-9b4d-ed835896273f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.040905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42714ac1-a579-4562-9b4d-ed835896273f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.088469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-42714ac1-a579-4562-9b4d-ed835896273f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:58.090548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-42714ac1-a579-4562-9b4d-ed835896273f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:58.208711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1348/2689] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:30:58.211368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fce2b-9b64-499f-af1d-a70cccf5c0cd None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.212210 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fce2b-9b64-499f-af1d-a70cccf5c0cd None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.212644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fce2b-9b64-499f-af1d-a70cccf5c0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.242880 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-f26fce2b-9b64-499f-af1d-a70cccf5c0cd None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:58.246378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f26fce2b-9b64-499f-af1d-a70cccf5c0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:58.259644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1342/2690] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:58.261019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42714ac1-a579-4562-9b4d-ed835896273f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.272652 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42714ac1-a579-4562-9b4d-ed835896273f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.277908 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.279494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.279774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.280045 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.280816 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:30:58.281123 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.282867 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.300664 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1349/2691] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:30:58.305207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fce2b-9b64-499f-af1d-a70cccf5c0cd None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.315678 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fce2b-9b64-499f-af1d-a70cccf5c0cd None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.324171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d630a4-bb48-4976-8306-6aa57bef0846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.324865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d630a4-bb48-4976-8306-6aa57bef0846 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.325207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d630a4-bb48-4976-8306-6aa57bef0846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.325546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d630a4-bb48-4976-8306-6aa57bef0846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.326307 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d630a4-bb48-4976-8306-6aa57bef0846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.328157 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d630a4-bb48-4976-8306-6aa57bef0846 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.374097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1343/2692] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:30:58 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:30:58.378830 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.399905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a611d2-6a90-422f-979a-1fb0596fcbe1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.484031 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6843d33-e772-46c6-9222-4b81f36e9e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.484724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843d33-e772-46c6-9222-4b81f36e9e95 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.485385 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843d33-e772-46c6-9222-4b81f36e9e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.490440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6843d33-e772-46c6-9222-4b81f36e9e95 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.490440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6843d33-e772-46c6-9222-4b81f36e9e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.490440 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6843d33-e772-46c6-9222-4b81f36e9e95 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.516030 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1350/2693] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:30:58 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:58.520546 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d630a4-bb48-4976-8306-6aa57bef0846 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.532390 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d630a4-bb48-4976-8306-6aa57bef0846 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.537911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b82bf7-58b3-4c4e-97f1-1caccb38a124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.538528 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82bf7-58b3-4c4e-97f1-1caccb38a124 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.538812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82bf7-58b3-4c4e-97f1-1caccb38a124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.539230 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b82bf7-58b3-4c4e-97f1-1caccb38a124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.540040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b82bf7-58b3-4c4e-97f1-1caccb38a124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.541674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b82bf7-58b3-4c4e-97f1-1caccb38a124 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.555664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1344/2694] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:58.559497 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6843d33-e772-46c6-9222-4b81f36e9e95 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.570383 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6843d33-e772-46c6-9222-4b81f36e9e95 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.575372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd8c08-d519-46b2-9ef4-588dadea3897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.576053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd8c08-d519-46b2-9ef4-588dadea3897 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.576333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd8c08-d519-46b2-9ef4-588dadea3897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.576610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd8c08-d519-46b2-9ef4-588dadea3897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.577389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd8c08-d519-46b2-9ef4-588dadea3897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.579163 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd8c08-d519-46b2-9ef4-588dadea3897 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.592772 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1351/2695] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:58.598535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b82bf7-58b3-4c4e-97f1-1caccb38a124 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.609892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b82bf7-58b3-4c4e-97f1-1caccb38a124 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.615810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5153f-d5a1-4b81-a044-67b033f3bb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.616784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5153f-d5a1-4b81-a044-67b033f3bb26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.617135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5153f-d5a1-4b81-a044-67b033f3bb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.617562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5153f-d5a1-4b81-a044-67b033f3bb26 admin admin] PATH_INFO: `/v3/projects/51b945db1e8e44c19a459cacc98e1620/users/98bbc0c2241b4d539b498e6205074208/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.639043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5153f-d5a1-4b81-a044-67b033f3bb26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b945db1e8e44c19a459cacc98e1620, user_id=98bbc0c2241b4d539b498e6205074208, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.642114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5153f-d5a1-4b81-a044-67b033f3bb26 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.669485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1345/2696] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:30:58 2022] PUT /identity/v3/projects/51b945db1e8e44c19a459cacc98e1620/users/98bbc0c2241b4d539b498e6205074208/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:30:58.684563 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e4a35-7102-42e0-b1da-b7a486b2c46f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.684863 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e4a35-7102-42e0-b1da-b7a486b2c46f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.687448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e4a35-7102-42e0-b1da-b7a486b2c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.721124 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8b9e4a35-7102-42e0-b1da-b7a486b2c46f None None] MFA Rules not processed for user `98bbc0c2241b4d539b498e6205074208`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:58.722941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b9e4a35-7102-42e0-b1da-b7a486b2c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:58.762068 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:58.819796 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1352/2697] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:58.824573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5153f-d5a1-4b81-a044-67b033f3bb26 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.891932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5153f-d5a1-4b81-a044-67b033f3bb26 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:58.897559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200ec48-f99f-4c0f-9fc4-f0e1a529f948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:58.898243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ec48-f99f-4c0f-9fc4-f0e1a529f948 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:58.898525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ec48-f99f-4c0f-9fc4-f0e1a529f948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:58.898802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ec48-f99f-4c0f-9fc4-f0e1a529f948 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:58.899503 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200ec48-f99f-4c0f-9fc4-f0e1a529f948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:58.901523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200ec48-f99f-4c0f-9fc4-f0e1a529f948 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:58.930686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1346/2698] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:30:58 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:58.934715 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e4a35-7102-42e0-b1da-b7a486b2c46f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:58.945799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e4a35-7102-42e0-b1da-b7a486b2c46f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.005824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4175b0-7d01-417e-bfb9-7e41faad8ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.006505 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4175b0-7d01-417e-bfb9-7e41faad8ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.006789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4175b0-7d01-417e-bfb9-7e41faad8ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.007066 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4175b0-7d01-417e-bfb9-7e41faad8ac3 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.007767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4175b0-7d01-417e-bfb9-7e41faad8ac3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.009730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4175b0-7d01-417e-bfb9-7e41faad8ac3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.034855 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1353/2699] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:30:58 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:30:59.039098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200ec48-f99f-4c0f-9fc4-f0e1a529f948 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.050416 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200ec48-f99f-4c0f-9fc4-f0e1a529f948 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.055893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcf538-9b4c-4e46-b93c-63c407c596b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.056584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf538-9b4c-4e46-b93c-63c407c596b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.056865 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf538-9b4c-4e46-b93c-63c407c596b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.057136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edcf538-9b4c-4e46-b93c-63c407c596b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.057911 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcf538-9b4c-4e46-b93c-63c407c596b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.059507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edcf538-9b4c-4e46-b93c-63c407c596b3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.071531 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1347/2700] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:59.075861 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4175b0-7d01-417e-bfb9-7e41faad8ac3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.087616 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4175b0-7d01-417e-bfb9-7e41faad8ac3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.093815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bf0b8-4852-4f71-8685-e12c52701f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.094737 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bf0b8-4852-4f71-8685-e12c52701f3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.095017 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bf0b8-4852-4f71-8685-e12c52701f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.095301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3bf0b8-4852-4f71-8685-e12c52701f3e admin admin] PATH_INFO: `/v3/projects/9f2580b24f6a41628b57f0005ef269ca/users/cea58d399df84c22870c4247fc9e3262/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.112904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bf0b8-4852-4f71-8685-e12c52701f3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2580b24f6a41628b57f0005ef269ca, user_id=cea58d399df84c22870c4247fc9e3262, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.115839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3bf0b8-4852-4f71-8685-e12c52701f3e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.130518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1354/2701] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:30:59 2022] PUT /identity/v3/projects/9f2580b24f6a41628b57f0005ef269ca/users/cea58d399df84c22870c4247fc9e3262/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:59.134538 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcf538-9b4c-4e46-b93c-63c407c596b3 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.145426 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edcf538-9b4c-4e46-b93c-63c407c596b3 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.211649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:59.221761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6e38c-52d3-4568-bf7f-928d150b5669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.223003 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6e38c-52d3-4568-bf7f-928d150b5669 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.223556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6e38c-52d3-4568-bf7f-928d150b5669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.223963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6e38c-52d3-4568-bf7f-928d150b5669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.225070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6e38c-52d3-4568-bf7f-928d150b5669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.226906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6e38c-52d3-4568-bf7f-928d150b5669 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.238071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1348/2702] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:30:59.242697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bf0b8-4852-4f71-8685-e12c52701f3e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.254151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3bf0b8-4852-4f71-8685-e12c52701f3e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.259569 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.260466 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.260746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.261004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] PATH_INFO: `/v3/projects/9f2580b24f6a41628b57f0005ef269ca/users/cea58d399df84c22870c4247fc9e3262/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.263169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2580b24f6a41628b57f0005ef269ca, user_id=cea58d399df84c22870c4247fc9e3262, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.265842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.276837 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1355/2703] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:30:59 2022] PUT /identity/v3/projects/9f2580b24f6a41628b57f0005ef269ca/users/cea58d399df84c22870c4247fc9e3262/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:30:59.294509 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae1342-e7c5-4be5-b41c-6ad3271f25cc None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.295031 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae1342-e7c5-4be5-b41c-6ad3271f25cc None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.295574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae1342-e7c5-4be5-b41c-6ad3271f25cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.326217 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-26ae1342-e7c5-4be5-b41c-6ad3271f25cc None None] MFA Rules not processed for user `cea58d399df84c22870c4247fc9e3262`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:30:59.327337 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-26ae1342-e7c5-4be5-b41c-6ad3271f25cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:59.410190 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1349/2704] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:30:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:30:59.421429 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:30:59.424623 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:30:59.486503 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685406a-f74e-41a8-836c-9d72bd2c3924 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:30:59.492513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c69606-b203-4bb7-8bf5-3168a036c6b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:30:59.493482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c69606-b203-4bb7-8bf5-3168a036c6b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:30:59.493672 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c69606-b203-4bb7-8bf5-3168a036c6b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:30:59.493958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c69606-b203-4bb7-8bf5-3168a036c6b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:30:59.530538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:30:59.567874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c69606-b203-4bb7-8bf5-3168a036c6b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:30:59.570469 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c69606-b203-4bb7-8bf5-3168a036c6b8 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:30:59.587930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1356/2705] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:30:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:03.907432 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae1342-e7c5-4be5-b41c-6ad3271f25cc None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:03.910187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-26ae1342-e7c5-4be5-b41c-6ad3271f25cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:03.924507 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae1342-e7c5-4be5-b41c-6ad3271f25cc None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:03.931721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8873c27-d78d-47ea-bf5d-65b41ac7bb3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:03.931721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873c27-d78d-47ea-bf5d-65b41ac7bb3d service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:03.931721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873c27-d78d-47ea-bf5d-65b41ac7bb3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:03.931721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8873c27-d78d-47ea-bf5d-65b41ac7bb3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:03.943654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8873c27-d78d-47ea-bf5d-65b41ac7bb3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:03.945125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8873c27-d78d-47ea-bf5d-65b41ac7bb3d service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:03.958279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1350/2706] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:08.947344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c69606-b203-4bb7-8bf5-3168a036c6b8 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.016008 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c69606-b203-4bb7-8bf5-3168a036c6b8 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.021279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.022177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.022451 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.022730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] PATH_INFO: `/v3/users/98bbc0c2241b4d539b498e6205074208` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.023996 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98bbc0c2241b4d539b498e6205074208)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.025970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.075765 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98bbc0c2241b4d539b498e6205074208'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.076370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98bbc0c2241b4d539b498e6205074208'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.077007 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98bbc0c2241b4d539b498e6205074208'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.092915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98bbc0c2241b4d539b498e6205074208'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.102322 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98bbc0c2241b4d539b498e6205074208'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.116427 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1357/2707] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:31:08 2022] DELETE /identity/v3/users/98bbc0c2241b4d539b498e6205074208 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:09.121105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd393a9d-9ebd-4c18-bb4d-3ce19380385c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.121676 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd393a9d-9ebd-4c18-bb4d-3ce19380385c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.121909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd393a9d-9ebd-4c18-bb4d-3ce19380385c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.140663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-dd393a9d-9ebd-4c18-bb4d-3ce19380385c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:09.141893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dd393a9d-9ebd-4c18-bb4d-3ce19380385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:09.208489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1351/2708] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:09.218815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.220464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:09.288977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c87ec6-cb9f-4bf5-9a6c-5954e7b7cc15 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.292915 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfee022-7084-4e45-8657-3ab05fab6059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.293612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfee022-7084-4e45-8657-3ab05fab6059 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.293897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfee022-7084-4e45-8657-3ab05fab6059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.294170 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfee022-7084-4e45-8657-3ab05fab6059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.303096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:09.308794 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfee022-7084-4e45-8657-3ab05fab6059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.311202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfee022-7084-4e45-8657-3ab05fab6059 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.323344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1358/2709] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:09.635111 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd393a9d-9ebd-4c18-bb4d-3ce19380385c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.637586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dd393a9d-9ebd-4c18-bb4d-3ce19380385c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:09.654644 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd393a9d-9ebd-4c18-bb4d-3ce19380385c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.661370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.662269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.662556 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.662830 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] PATH_INFO: `/v3/projects/51b945db1e8e44c19a459cacc98e1620` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.664113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b945db1e8e44c19a459cacc98e1620)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.666243 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.722589 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b945db1e8e44c19a459cacc98e1620', 'request_id': 'req-541d7f56-0577-4ef3-8041-e2adbf2cc887'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.735744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b945db1e8e44c19a459cacc98e1620', 'request_id': 'req-541d7f56-0577-4ef3-8041-e2adbf2cc887'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.744246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b945db1e8e44c19a459cacc98e1620', 'request_id': 'req-541d7f56-0577-4ef3-8041-e2adbf2cc887'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.745572 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:09.757885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] The token cache is being invalidate because project 51b945db1e8e44c19a459cacc98e1620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:31:09.758367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.759258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.761076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1352/2710] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:31:09 2022] DELETE /identity/v3/projects/51b945db1e8e44c19a459cacc98e1620 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:09.765865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfee022-7084-4e45-8657-3ab05fab6059 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:09.777588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfee022-7084-4e45-8657-3ab05fab6059 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:09.847651 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:09.848580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:09.848841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:09.849112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] PATH_INFO: `/v3/users/cea58d399df84c22870c4247fc9e3262` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:09.850924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea58d399df84c22870c4247fc9e3262)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:09.852428 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:09.904576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea58d399df84c22870c4247fc9e3262'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.905141 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea58d399df84c22870c4247fc9e3262'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.905754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea58d399df84c22870c4247fc9e3262'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.920918 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea58d399df84c22870c4247fc9e3262'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.933439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c9046b38-da52-41df-8e54-96db0221888f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea58d399df84c22870c4247fc9e3262'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:09.947452 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1359/2711] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:31:09 2022] DELETE /identity/v3/users/cea58d399df84c22870c4247fc9e3262 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:10.249679 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.251154 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:10.265302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d7f56-0577-4ef3-8041-e2adbf2cc887 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.335489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.336379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.336656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.336939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] PATH_INFO: `/v3/projects/9f2580b24f6a41628b57f0005ef269ca` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.338169 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f2580b24f6a41628b57f0005ef269ca)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.340280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.396445 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f2580b24f6a41628b57f0005ef269ca', 'request_id': 'req-70ce5d0e-4772-459e-8c6c-b1dde729642f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:10.407060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f2580b24f6a41628b57f0005ef269ca', 'request_id': 'req-70ce5d0e-4772-459e-8c6c-b1dde729642f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:10.414054 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f2580b24f6a41628b57f0005ef269ca', 'request_id': 'req-70ce5d0e-4772-459e-8c6c-b1dde729642f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:10.425498 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] The token cache is being invalidate because project 9f2580b24f6a41628b57f0005ef269ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:31:10.425802 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:10.426488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ce5d0e-4772-459e-8c6c-b1dde729642f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:10.428080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1353/2712] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:31:10 2022] DELETE /identity/v3/projects/9f2580b24f6a41628b57f0005ef269ca => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:10.434106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb85d77-b2cf-4683-853a-601d2248983d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.434393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb85d77-b2cf-4683-853a-601d2248983d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.434670 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb85d77-b2cf-4683-853a-601d2248983d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.452834 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5bb85d77-b2cf-4683-853a-601d2248983d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:10.454109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5bb85d77-b2cf-4683-853a-601d2248983d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:10.522321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1360/2713] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:10.530906 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589fad-c4dc-4947-a4a6-9ee8005c12c4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.531196 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589fad-c4dc-4947-a4a6-9ee8005c12c4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.531479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1589fad-c4dc-4947-a4a6-9ee8005c12c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.551285 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-b1589fad-c4dc-4947-a4a6-9ee8005c12c4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:10.552585 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b1589fad-c4dc-4947-a4a6-9ee8005c12c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:10.561339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1354/2714] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:10.565689 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb85d77-b2cf-4683-853a-601d2248983d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.576353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb85d77-b2cf-4683-853a-601d2248983d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.582606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.583231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.583512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.583784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.584524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:10.584832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.586474 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.596082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:10.600534 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1361/2715] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:10.604919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1589fad-c4dc-4947-a4a6-9ee8005c12c4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.616686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1589fad-c4dc-4947-a4a6-9ee8005c12c4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.625127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcd7de-d4f0-4b5d-b124-06f87a100157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.626205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcd7de-d4f0-4b5d-b124-06f87a100157 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.626517 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcd7de-d4f0-4b5d-b124-06f87a100157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.627033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcd7de-d4f0-4b5d-b124-06f87a100157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.628020 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcd7de-d4f0-4b5d-b124-06f87a100157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.631377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcd7de-d4f0-4b5d-b124-06f87a100157 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.666848 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1355/2716] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:31:10 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:10.671454 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.683571 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb07cd-c9f9-4c97-93c6-420f8f9a5234 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.749882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59decfa-3c6f-4163-9bac-0f151e82df42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.750565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59decfa-3c6f-4163-9bac-0f151e82df42 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.750850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59decfa-3c6f-4163-9bac-0f151e82df42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.751125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59decfa-3c6f-4163-9bac-0f151e82df42 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.751822 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59decfa-3c6f-4163-9bac-0f151e82df42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.753852 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59decfa-3c6f-4163-9bac-0f151e82df42 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.781538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1362/2717] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:31:10 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:31:10.785885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcd7de-d4f0-4b5d-b124-06f87a100157 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.796485 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcd7de-d4f0-4b5d-b124-06f87a100157 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.801698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cf53f-40ac-4d2f-bd6e-dafa632f55e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.802352 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf53f-40ac-4d2f-bd6e-dafa632f55e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.802636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf53f-40ac-4d2f-bd6e-dafa632f55e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.802914 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139cf53f-40ac-4d2f-bd6e-dafa632f55e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.803655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cf53f-40ac-4d2f-bd6e-dafa632f55e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.805302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139cf53f-40ac-4d2f-bd6e-dafa632f55e1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.816879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1356/2718] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:10.820991 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59decfa-3c6f-4163-9bac-0f151e82df42 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.835611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59decfa-3c6f-4163-9bac-0f151e82df42 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.843418 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634644d6-69ac-4b78-9f62-a35f0eb6fe23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.844389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634644d6-69ac-4b78-9f62-a35f0eb6fe23 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.844859 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634644d6-69ac-4b78-9f62-a35f0eb6fe23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.845372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634644d6-69ac-4b78-9f62-a35f0eb6fe23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.846572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634644d6-69ac-4b78-9f62-a35f0eb6fe23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.848766 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634644d6-69ac-4b78-9f62-a35f0eb6fe23 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.860072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1363/2719] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:10.865840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cf53f-40ac-4d2f-bd6e-dafa632f55e1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:10.877393 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139cf53f-40ac-4d2f-bd6e-dafa632f55e1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:10.883232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:10.884135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.884413 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.884693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] PATH_INFO: `/v3/projects/777b256684194f5e88bd1feada16c980/users/b953b966a8514fbb82b251c2f854f285/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.901282 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777b256684194f5e88bd1feada16c980, user_id=b953b966a8514fbb82b251c2f854f285, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:10.903936 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:10.919969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1357/2720] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:31:10 2022] PUT /identity/v3/projects/777b256684194f5e88bd1feada16c980/users/b953b966a8514fbb82b251c2f854f285/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:10.931401 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ed4ae-6141-480e-92cc-d9e12ca31922 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:10.931721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ed4ae-6141-480e-92cc-d9e12ca31922 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:10.932006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ed4ae-6141-480e-92cc-d9e12ca31922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:10.961291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f33ed4ae-6141-480e-92cc-d9e12ca31922 None None] MFA Rules not processed for user `b953b966a8514fbb82b251c2f854f285`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:10.962522 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f33ed4ae-6141-480e-92cc-d9e12ca31922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:11.031711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1364/2721] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:11.043448 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:11.045516 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:11.066131 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:11.114641 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d3cac-58b4-494a-b3e2-9271f65c9405 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:11.118825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f53109-9ec3-429c-b01d-2677c18f91d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db842931b2394e8684012057347080af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:11.119482 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f53109-9ec3-429c-b01d-2677c18f91d3 service glance] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:11.119767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f53109-9ec3-429c-b01d-2677c18f91d3 service glance] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:11.120042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f53109-9ec3-429c-b01d-2677c18f91d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:11.129571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f53109-9ec3-429c-b01d-2677c18f91d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:11.131022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f53109-9ec3-429c-b01d-2677c18f91d3 service glance] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:11.141425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1358/2722] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:21.028352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:21.136197 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=19 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:24.753205 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ed4ae-6141-480e-92cc-d9e12ca31922 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:24.762656 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f33ed4ae-6141-480e-92cc-d9e12ca31922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:24.827177 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ed4ae-6141-480e-92cc-d9e12ca31922 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:24.831980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be9786-adf6-41d7-a645-5406ab9b1263 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:24.832782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9786-adf6-41d7-a645-5406ab9b1263 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:24.833072 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9786-adf6-41d7-a645-5406ab9b1263 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:24.833378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be9786-adf6-41d7-a645-5406ab9b1263 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:24.835897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a6be9786-adf6-41d7-a645-5406ab9b1263 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:24.900749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be9786-adf6-41d7-a645-5406ab9b1263 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:24.902461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be9786-adf6-41d7-a645-5406ab9b1263 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:24.912642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1365/2723] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:28.857512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f53109-9ec3-429c-b01d-2677c18f91d3 service glance] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:28.933219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f53109-9ec3-429c-b01d-2677c18f91d3 service glance] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:28.938464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:28.939354 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:28.939633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:28.939913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] PATH_INFO: `/v3/users/b953b966a8514fbb82b251c2f854f285` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:28.941491 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b953b966a8514fbb82b251c2f854f285)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:28.943405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.002036 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b953b966a8514fbb82b251c2f854f285'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.020607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b953b966a8514fbb82b251c2f854f285'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.021820 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b953b966a8514fbb82b251c2f854f285'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.030849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b953b966a8514fbb82b251c2f854f285'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.038667 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b953b966a8514fbb82b251c2f854f285'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.048692 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1359/2724] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:31:28 2022] DELETE /identity/v3/users/b953b966a8514fbb82b251c2f854f285 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:29.050587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212718-a598-4e44-948f-3935df0ef86c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.051088 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212718-a598-4e44-948f-3935df0ef86c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.051197 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212718-a598-4e44-948f-3935df0ef86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.068839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-11212718-a598-4e44-948f-3935df0ef86c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:29.070078 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-11212718-a598-4e44-948f-3935df0ef86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:29.147456 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1366/2725] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:29.157602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.159249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:29.231038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36797e-50db-42ee-9150-d6284f34e032 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.235200 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1ae9dc-0713-4a2a-b161-59aa53e3dd54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.235890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ae9dc-0713-4a2a-b161-59aa53e3dd54 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.236171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ae9dc-0713-4a2a-b161-59aa53e3dd54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.236446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1ae9dc-0713-4a2a-b161-59aa53e3dd54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.247278 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ae9dc-0713-4a2a-b161-59aa53e3dd54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.248708 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1ae9dc-0713-4a2a-b161-59aa53e3dd54 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.262624 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1360/2726] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:29.555346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11212718-a598-4e44-948f-3935df0ef86c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.556938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-11212718-a598-4e44-948f-3935df0ef86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:29.571069 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11212718-a598-4e44-948f-3935df0ef86c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.577020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.577973 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.578439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.578720 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] PATH_INFO: `/v3/projects/777b256684194f5e88bd1feada16c980` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.580166 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] RBAC: Authorizing `identity:delete_project(project_id=777b256684194f5e88bd1feada16c980)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.582580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.637683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777b256684194f5e88bd1feada16c980', 'request_id': 'req-dab259bc-a580-4837-b9d4-579339bc3235'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.639928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:29.645827 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777b256684194f5e88bd1feada16c980', 'request_id': 'req-dab259bc-a580-4837-b9d4-579339bc3235'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.677711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777b256684194f5e88bd1feada16c980', 'request_id': 'req-dab259bc-a580-4837-b9d4-579339bc3235'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.697292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] The token cache is being invalidate because project 777b256684194f5e88bd1feada16c980 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:31:29.697555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.698328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-dab259bc-a580-4837-b9d4-579339bc3235 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:29.700686 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1367/2727] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:31:29 2022] DELETE /identity/v3/projects/777b256684194f5e88bd1feada16c980 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:29.708514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf460d4-9327-4850-bd0e-4c0d84c50f9d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.708838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf460d4-9327-4850-bd0e-4c0d84c50f9d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.709083 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf460d4-9327-4850-bd0e-4c0d84c50f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.730713 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ccf460d4-9327-4850-bd0e-4c0d84c50f9d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:29.732279 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ccf460d4-9327-4850-bd0e-4c0d84c50f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:29.757130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:29.809021 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1361/2728] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:29.816911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2437b2-7eb4-42cb-b03c-276d9d7d8c94 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.817280 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2437b2-7eb4-42cb-b03c-276d9d7d8c94 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.817565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2437b2-7eb4-42cb-b03c-276d9d7d8c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.838356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0c2437b2-7eb4-42cb-b03c-276d9d7d8c94 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:29.839588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c2437b2-7eb4-42cb-b03c-276d9d7d8c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:29.848039 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1368/2729] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:29.852471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf460d4-9327-4850-bd0e-4c0d84c50f9d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.862225 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf460d4-9327-4850-bd0e-4c0d84c50f9d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.867574 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.868238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.868521 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.868799 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.869584 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:29.869897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.871515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.884091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1362/2730] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:29.888611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2437b2-7eb4-42cb-b03c-276d9d7d8c94 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.898943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2437b2-7eb4-42cb-b03c-276d9d7d8c94 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:29.904660 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453217bf-369a-4b82-945a-8cd913a9663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:29.905381 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453217bf-369a-4b82-945a-8cd913a9663f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:29.905673 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453217bf-369a-4b82-945a-8cd913a9663f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:29.905957 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453217bf-369a-4b82-945a-8cd913a9663f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:29.906668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453217bf-369a-4b82-945a-8cd913a9663f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:29.908635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453217bf-369a-4b82-945a-8cd913a9663f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:29.942642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1369/2731] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:31:29 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:29.943787 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:29.955158 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfa421-ee57-4b6f-a30a-8e4ac40b31fb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.020115 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d429d-33cb-4000-bd9a-ba4506f9faec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.020794 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d429d-33cb-4000-bd9a-ba4506f9faec admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.021078 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d429d-33cb-4000-bd9a-ba4506f9faec admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.021391 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d429d-33cb-4000-bd9a-ba4506f9faec admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.022066 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d429d-33cb-4000-bd9a-ba4506f9faec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.024339 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d429d-33cb-4000-bd9a-ba4506f9faec admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.052845 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1363/2732] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:31:29 2022] POST /identity/v3/users => generated 281 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:30.057264 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453217bf-369a-4b82-945a-8cd913a9663f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:30.069103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453217bf-369a-4b82-945a-8cd913a9663f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.074565 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776cb1-0e2a-4717-8e04-55fe31e0f283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.075231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776cb1-0e2a-4717-8e04-55fe31e0f283 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.075513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776cb1-0e2a-4717-8e04-55fe31e0f283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.075795 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e776cb1-0e2a-4717-8e04-55fe31e0f283 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.076605 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e776cb1-0e2a-4717-8e04-55fe31e0f283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.078252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e776cb1-0e2a-4717-8e04-55fe31e0f283 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.089622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1370/2733] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:30.095353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d429d-33cb-4000-bd9a-ba4506f9faec admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:30.108412 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d429d-33cb-4000-bd9a-ba4506f9faec admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.114562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb796843-d805-4b84-998b-4bf7194167d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.115248 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb796843-d805-4b84-998b-4bf7194167d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.115532 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb796843-d805-4b84-998b-4bf7194167d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.115811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb796843-d805-4b84-998b-4bf7194167d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.116561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb796843-d805-4b84-998b-4bf7194167d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.118203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb796843-d805-4b84-998b-4bf7194167d9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.129335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1364/2734] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:30.133894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776cb1-0e2a-4717-8e04-55fe31e0f283 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:30.148004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e776cb1-0e2a-4717-8e04-55fe31e0f283 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.153352 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.154248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.154527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.154797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] PATH_INFO: `/v3/projects/0b093ec22ff54b73b051a09a59ec93cb/users/5544b578255a47baa3c6f0f54fa02dc6/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.174136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b093ec22ff54b73b051a09a59ec93cb, user_id=5544b578255a47baa3c6f0f54fa02dc6, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.177220 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.199014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1371/2735] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:31:30 2022] PUT /identity/v3/projects/0b093ec22ff54b73b051a09a59ec93cb/users/5544b578255a47baa3c6f0f54fa02dc6/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:30.210364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9002-0425-414f-895f-9081d5746e45 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.210723 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9002-0425-414f-895f-9081d5746e45 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.211167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e9002-0425-414f-895f-9081d5746e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.246303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-2e4e9002-0425-414f-895f-9081d5746e45 None None] MFA Rules not processed for user `5544b578255a47baa3c6f0f54fa02dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:30.247553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2e4e9002-0425-414f-895f-9081d5746e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:30.314835 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1365/2736] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:30.348836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:30.357817 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:30.434127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a8f457-aa64-4f2e-a62d-23ee82389f9c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.441085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e815f3f-e008-4225-993c-97724a152a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.441085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e815f3f-e008-4225-993c-97724a152a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.441085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e815f3f-e008-4225-993c-97724a152a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.441085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e815f3f-e008-4225-993c-97724a152a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.448891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e815f3f-e008-4225-993c-97724a152a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.450510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e815f3f-e008-4225-993c-97724a152a7b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.460366 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1372/2737] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:31:30.474434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e9002-0425-414f-895f-9081d5746e45 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:30.495651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:30.542479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e9002-0425-414f-895f-9081d5746e45 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.547310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.548235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.548514 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.548789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] PATH_INFO: `/v3/users/5544b578255a47baa3c6f0f54fa02dc6` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.550062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5544b578255a47baa3c6f0f54fa02dc6)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.552259 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.605213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5544b578255a47baa3c6f0f54fa02dc6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:30.619277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5544b578255a47baa3c6f0f54fa02dc6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:30.620042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5544b578255a47baa3c6f0f54fa02dc6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:30.627630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5544b578255a47baa3c6f0f54fa02dc6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:30.637932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5544b578255a47baa3c6f0f54fa02dc6'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:30.644776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1366/2738] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:31:30 2022] DELETE /identity/v3/users/5544b578255a47baa3c6f0f54fa02dc6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:30.649480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137bda-144f-4c1d-8414-a4a7f22eb143 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.649783 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137bda-144f-4c1d-8414-a4a7f22eb143 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.650057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d137bda-144f-4c1d-8414-a4a7f22eb143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.668655 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2d137bda-144f-4c1d-8414-a4a7f22eb143 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:30.669901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d137bda-144f-4c1d-8414-a4a7f22eb143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:30.751770 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1373/2739] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:30.762038 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:30.763594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:30.831558 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea0851-960a-44c9-8e55-84005e31b00a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:30.835710 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcf527-3d13-489e-85d1-a110129177b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:30.836367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcf527-3d13-489e-85d1-a110129177b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:30.836649 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcf527-3d13-489e-85d1-a110129177b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:30.836933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcf527-3d13-489e-85d1-a110129177b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:30.849134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcf527-3d13-489e-85d1-a110129177b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:30.850693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcf527-3d13-489e-85d1-a110129177b2 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:30.863149 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1367/2740] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:31.176803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d137bda-144f-4c1d-8414-a4a7f22eb143 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.178384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d137bda-144f-4c1d-8414-a4a7f22eb143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:31.191900 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d137bda-144f-4c1d-8414-a4a7f22eb143 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.196582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.197493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.197779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.198059 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] PATH_INFO: `/v3/projects/0b093ec22ff54b73b051a09a59ec93cb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.199291 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b093ec22ff54b73b051a09a59ec93cb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.201393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.214262 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:31.257587 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b093ec22ff54b73b051a09a59ec93cb', 'request_id': 'req-b6f7dccc-868a-4e48-a143-234a352b4e52'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:31.267309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b093ec22ff54b73b051a09a59ec93cb', 'request_id': 'req-b6f7dccc-868a-4e48-a143-234a352b4e52'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:31.276798 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b093ec22ff54b73b051a09a59ec93cb', 'request_id': 'req-b6f7dccc-868a-4e48-a143-234a352b4e52'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:31.289057 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] The token cache is being invalidate because project 0b093ec22ff54b73b051a09a59ec93cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:31:31.289384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:31.290053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b6f7dccc-868a-4e48-a143-234a352b4e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:31.291634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1374/2741] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:31:31 2022] DELETE /identity/v3/projects/0b093ec22ff54b73b051a09a59ec93cb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:31.296651 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d0c79-a7ed-4602-ac66-4961d2cee7c1 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.296932 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d0c79-a7ed-4602-ac66-4961d2cee7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.297238 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d0c79-a7ed-4602-ac66-4961d2cee7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.316534 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-ac2d0c79-a7ed-4602-ac66-4961d2cee7c1 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:31.317984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-ac2d0c79-a7ed-4602-ac66-4961d2cee7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:31.354815 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:31.384672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1368/2742] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:31.392744 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ef3f-37cb-484c-a582-dc1f7ad2dcc8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.393038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ef3f-37cb-484c-a582-dc1f7ad2dcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.393382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ef3f-37cb-484c-a582-dc1f7ad2dcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.412752 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b322ef3f-37cb-484c-a582-dc1f7ad2dcc8 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:31.413992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b322ef3f-37cb-484c-a582-dc1f7ad2dcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:31.422658 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1375/2743] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:31.435025 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d0c79-a7ed-4602-ac66-4961d2cee7c1 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.445748 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d0c79-a7ed-4602-ac66-4961d2cee7c1 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.458140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.459203 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.459721 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.460135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.461431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80239) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:31.461994 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.464876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.480105 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1369/2744] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:31.484881 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322ef3f-37cb-484c-a582-dc1f7ad2dcc8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.495645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322ef3f-37cb-484c-a582-dc1f7ad2dcc8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.501590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6fc30-bc1d-4b68-89b0-3c9e4dab44ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.502270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6fc30-bc1d-4b68-89b0-3c9e4dab44ed admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.502532 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6fc30-bc1d-4b68-89b0-3c9e4dab44ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.502810 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da6fc30-bc1d-4b68-89b0-3c9e4dab44ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.504038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6fc30-bc1d-4b68-89b0-3c9e4dab44ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.505638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da6fc30-bc1d-4b68-89b0-3c9e4dab44ed admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.534870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1376/2745] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:31:31 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:31.539863 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.551817 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebc4dd6-4d46-40e3-8f10-64cbc36f4d55 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.615467 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f12a05-f5d0-4894-85b9-43a2b1576274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.616144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f12a05-f5d0-4894-85b9-43a2b1576274 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.616425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f12a05-f5d0-4894-85b9-43a2b1576274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.616698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f12a05-f5d0-4894-85b9-43a2b1576274 admin admin] PATH_INFO: `/v3/users` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.617440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f12a05-f5d0-4894-85b9-43a2b1576274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.619340 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f12a05-f5d0-4894-85b9-43a2b1576274 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.643743 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1370/2746] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:31:31 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:31.648136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6fc30-bc1d-4b68-89b0-3c9e4dab44ed admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.660079 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da6fc30-bc1d-4b68-89b0-3c9e4dab44ed admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.665713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9d45-d4d3-4f86-b858-3bb5f2441c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.666399 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9d45-d4d3-4f86-b858-3bb5f2441c2d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.666687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9d45-d4d3-4f86-b858-3bb5f2441c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.666970 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a9d45-d4d3-4f86-b858-3bb5f2441c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.667703 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9d45-d4d3-4f86-b858-3bb5f2441c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.669359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a9d45-d4d3-4f86-b858-3bb5f2441c2d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.684235 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1377/2747] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:31.688439 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f12a05-f5d0-4894-85b9-43a2b1576274 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.699975 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f12a05-f5d0-4894-85b9-43a2b1576274 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.705616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e95516-bc9a-4fc0-9423-8c1d6f9aa02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.706252 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e95516-bc9a-4fc0-9423-8c1d6f9aa02e admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.706564 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e95516-bc9a-4fc0-9423-8c1d6f9aa02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.707044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e95516-bc9a-4fc0-9423-8c1d6f9aa02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.708014 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e95516-bc9a-4fc0-9423-8c1d6f9aa02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.709674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e95516-bc9a-4fc0-9423-8c1d6f9aa02e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.720943 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1371/2748] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:31.725295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9d45-d4d3-4f86-b858-3bb5f2441c2d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.735842 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a9d45-d4d3-4f86-b858-3bb5f2441c2d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.741841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3cddb-f495-4eff-a6a3-907bce767252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.742763 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3cddb-f495-4eff-a6a3-907bce767252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.743047 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3cddb-f495-4eff-a6a3-907bce767252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.743321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3cddb-f495-4eff-a6a3-907bce767252 admin admin] PATH_INFO: `/v3/projects/6fb3bda4921d4cd1b384a66fbf71b33e/users/6da00a124fb64655b3ed3087c633d69d/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.764041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3cddb-f495-4eff-a6a3-907bce767252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb3bda4921d4cd1b384a66fbf71b33e, user_id=6da00a124fb64655b3ed3087c633d69d, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.766788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3cddb-f495-4eff-a6a3-907bce767252 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:31.783255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1378/2749] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:31:31 2022] PUT /identity/v3/projects/6fb3bda4921d4cd1b384a66fbf71b33e/users/6da00a124fb64655b3ed3087c633d69d/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:31.795297 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96162e-9118-4f86-ab79-43e6707397e4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.795660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96162e-9118-4f86-ab79-43e6707397e4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.795998 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96162e-9118-4f86-ab79-43e6707397e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.826846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-6a96162e-9118-4f86-ab79-43e6707397e4 None None] MFA Rules not processed for user `6da00a124fb64655b3ed3087c633d69d`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:31.828077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6a96162e-9118-4f86-ab79-43e6707397e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:31.861085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f8d5-0611-4034-b094-954ec72ddb9c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.861593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f8d5-0611-4034-b094-954ec72ddb9c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.862209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543f8d5-0611-4034-b094-954ec72ddb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.884105 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4543f8d5-0611-4034-b094-954ec72ddb9c None None] MFA Rules not processed for user `fbbafeb56a4d4fd282275e4198ede783`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:31.885372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4543f8d5-0611-4034-b094-954ec72ddb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:31.898493 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1372/2750] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:31.910926 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96162e-9118-4f86-ab79-43e6707397e4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:31.912428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-6a96162e-9118-4f86-ab79-43e6707397e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:31.952901 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1379/2751] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:31.956076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:31.975796 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96162e-9118-4f86-ab79-43e6707397e4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:31.984789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd92d3-bdcb-4403-9441-1cd221b43d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:31.985533 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd92d3-bdcb-4403-9441-1cd221b43d39 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:31.985816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd92d3-bdcb-4403-9441-1cd221b43d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:31.986090 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd92d3-bdcb-4403-9441-1cd221b43d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:31.996184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd92d3-bdcb-4403-9441-1cd221b43d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:31.997831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd92d3-bdcb-4403-9441-1cd221b43d39 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:32.008832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1373/2752] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:35.144976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543f8d5-0611-4034-b094-954ec72ddb9c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:35.203514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:35.229397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543f8d5-0611-4034-b094-954ec72ddb9c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:35.239187 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:35.240024 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:35.240297 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:35.240572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] PATH_INFO: `/v3/users/6da00a124fb64655b3ed3087c633d69d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:35.241894 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da00a124fb64655b3ed3087c633d69d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:35.243905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:35.324661 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da00a124fb64655b3ed3087c633d69d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:35.325513 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da00a124fb64655b3ed3087c633d69d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:35.326282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da00a124fb64655b3ed3087c633d69d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:35.361447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da00a124fb64655b3ed3087c633d69d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:35.375316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da00a124fb64655b3ed3087c633d69d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:35.396625 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1380/2753] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:31:35 2022] DELETE /identity/v3/users/6da00a124fb64655b3ed3087c633d69d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:35.403452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463496dd-e14e-4323-aac2-7bab2f487f58 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:35.404056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463496dd-e14e-4323-aac2-7bab2f487f58 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:35.404056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463496dd-e14e-4323-aac2-7bab2f487f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:35.429524 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-463496dd-e14e-4323-aac2-7bab2f487f58 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:35.431822 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-463496dd-e14e-4323-aac2-7bab2f487f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:35.537288 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1374/2754] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:35.548489 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:35.549987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:35.624343 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d54fc7-a16f-4ddf-9afa-c6b57f501b72 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:35.629561 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a44c9-7342-4087-ad1d-ef645bd032ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:35.630397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a44c9-7342-4087-ad1d-ef645bd032ac service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:35.631125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a44c9-7342-4087-ad1d-ef645bd032ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:35.631421 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a44c9-7342-4087-ad1d-ef645bd032ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:35.642145 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a44c9-7342-4087-ad1d-ef645bd032ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:35.643629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a44c9-7342-4087-ad1d-ef645bd032ac service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:35.654716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1381/2755] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:35.984876 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463496dd-e14e-4323-aac2-7bab2f487f58 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:35.986664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-463496dd-e14e-4323-aac2-7bab2f487f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:36.000909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463496dd-e14e-4323-aac2-7bab2f487f58 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.006887 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.008601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.008601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.008601 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] PATH_INFO: `/v3/projects/6fb3bda4921d4cd1b384a66fbf71b33e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.010145 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb3bda4921d4cd1b384a66fbf71b33e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.012450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.077058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb3bda4921d4cd1b384a66fbf71b33e', 'request_id': 'req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:36.086537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb3bda4921d4cd1b384a66fbf71b33e', 'request_id': 'req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:36.095257 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb3bda4921d4cd1b384a66fbf71b33e', 'request_id': 'req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:36.114471 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] The token cache is being invalidate because project 6fb3bda4921d4cd1b384a66fbf71b33e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:31:36.114754 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:36.115554 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-b50a4bd8-3f5d-4b0d-bce4-3b434539de28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:36.117661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1375/2756] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:31:35 2022] DELETE /identity/v3/projects/6fb3bda4921d4cd1b384a66fbf71b33e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:36.127370 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64b4ab-b12b-46f4-a49c-5bfc03d31fc7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.127674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64b4ab-b12b-46f4-a49c-5bfc03d31fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.127959 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64b4ab-b12b-46f4-a49c-5bfc03d31fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.147649 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9d64b4ab-b12b-46f4-a49c-5bfc03d31fc7 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:36.148880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9d64b4ab-b12b-46f4-a49c-5bfc03d31fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:36.235902 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1382/2757] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:36.244818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c289991-dc1e-4ff0-8a6c-98acb5075cba None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.245213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c289991-dc1e-4ff0-8a6c-98acb5075cba None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.245622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c289991-dc1e-4ff0-8a6c-98acb5075cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.268674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7c289991-dc1e-4ff0-8a6c-98acb5075cba None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:36.270034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7c289991-dc1e-4ff0-8a6c-98acb5075cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:36.279515 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1376/2758] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:36.283588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64b4ab-b12b-46f4-a49c-5bfc03d31fc7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.292493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:36.305394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64b4ab-b12b-46f4-a49c-5bfc03d31fc7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.311882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.312494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.312971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.313360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.314195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:31:36.314584 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.316479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.333400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1383/2759] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:31:36.334665 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c289991-dc1e-4ff0-8a6c-98acb5075cba None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.344378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c289991-dc1e-4ff0-8a6c-98acb5075cba None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.350627 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021616d9-ffcf-4b2d-a8cb-99300efeeee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.351312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021616d9-ffcf-4b2d-a8cb-99300efeeee1 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.351597 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021616d9-ffcf-4b2d-a8cb-99300efeeee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.351883 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021616d9-ffcf-4b2d-a8cb-99300efeeee1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.352596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021616d9-ffcf-4b2d-a8cb-99300efeeee1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.354559 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021616d9-ffcf-4b2d-a8cb-99300efeeee1 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.391832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1377/2760] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:31:36 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:36.393100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.406174 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467553be-24cc-4094-88d6-3cce2b4437fa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.478070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c848f7-d5a5-4f89-b093-aad43e13ae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.478960 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c848f7-d5a5-4f89-b093-aad43e13ae64 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.479139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c848f7-d5a5-4f89-b093-aad43e13ae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.479359 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c848f7-d5a5-4f89-b093-aad43e13ae64 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.480062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c848f7-d5a5-4f89-b093-aad43e13ae64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.482153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c848f7-d5a5-4f89-b093-aad43e13ae64 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.506470 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1384/2761] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:31:36 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:31:36.513479 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021616d9-ffcf-4b2d-a8cb-99300efeeee1 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.527580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021616d9-ffcf-4b2d-a8cb-99300efeeee1 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.533604 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa98d1-7bc4-413d-8cc5-0773276179a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.534810 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa98d1-7bc4-413d-8cc5-0773276179a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.535189 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa98d1-7bc4-413d-8cc5-0773276179a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.535522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa98d1-7bc4-413d-8cc5-0773276179a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.536295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa98d1-7bc4-413d-8cc5-0773276179a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.538024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa98d1-7bc4-413d-8cc5-0773276179a5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.553760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1378/2762] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:36.555129 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c848f7-d5a5-4f89-b093-aad43e13ae64 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.565958 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c848f7-d5a5-4f89-b093-aad43e13ae64 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.571238 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872298ab-ca3e-47bd-b9e6-a3e064b8b684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.572063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872298ab-ca3e-47bd-b9e6-a3e064b8b684 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.572260 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872298ab-ca3e-47bd-b9e6-a3e064b8b684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.572479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872298ab-ca3e-47bd-b9e6-a3e064b8b684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.573241 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872298ab-ca3e-47bd-b9e6-a3e064b8b684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.574898 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872298ab-ca3e-47bd-b9e6-a3e064b8b684 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.587127 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1385/2763] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:31:36.592617 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa98d1-7bc4-413d-8cc5-0773276179a5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.604629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa98d1-7bc4-413d-8cc5-0773276179a5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.609816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.610763 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.611178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.611178 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] PATH_INFO: `/v3/projects/31e992228e2640d78f4ef6b46209be8b/users/99b01856a4384dfa9102092cb6ccc3c1/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.629152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e992228e2640d78f4ef6b46209be8b, user_id=99b01856a4384dfa9102092cb6ccc3c1, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.631958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.645076 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:36.653140 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1379/2764] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:31:36 2022] PUT /identity/v3/projects/31e992228e2640d78f4ef6b46209be8b/users/99b01856a4384dfa9102092cb6ccc3c1/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:36.666099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad835ee-48b1-4e94-b36d-9d9539dd3f4f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.666749 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad835ee-48b1-4e94-b36d-9d9539dd3f4f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.667304 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad835ee-48b1-4e94-b36d-9d9539dd3f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.703551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0ad835ee-48b1-4e94-b36d-9d9539dd3f4f None None] MFA Rules not processed for user `99b01856a4384dfa9102092cb6ccc3c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:36.705277 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ad835ee-48b1-4e94-b36d-9d9539dd3f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:36.795865 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1386/2765] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 17:31:36.808851 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:36.811082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:36.881219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0fd52-c432-488a-a0ce-953470a5ab0b admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:36.885370 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b7430-5210-4ebf-93e0-845dfb629c51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:36.886043 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7430-5210-4ebf-93e0-845dfb629c51 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:36.886323 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7430-5210-4ebf-93e0-845dfb629c51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:36.886605 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7430-5210-4ebf-93e0-845dfb629c51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:36.899296 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b7430-5210-4ebf-93e0-845dfb629c51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:36.900783 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b7430-5210-4ebf-93e0-845dfb629c51 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:36.912114 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1380/2766] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:39.534063 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad835ee-48b1-4e94-b36d-9d9539dd3f4f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:39.620118 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad835ee-48b1-4e94-b36d-9d9539dd3f4f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:39.625887 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:39.627125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:39.627406 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:39.627674 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] PATH_INFO: `/v3/users/99b01856a4384dfa9102092cb6ccc3c1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:39.628974 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] RBAC: Authorizing `identity:delete_user(user_id=99b01856a4384dfa9102092cb6ccc3c1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:39.630921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:39.632444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:39.689606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99b01856a4384dfa9102092cb6ccc3c1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:39.690309 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99b01856a4384dfa9102092cb6ccc3c1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:39.691538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99b01856a4384dfa9102092cb6ccc3c1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:39.705146 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99b01856a4384dfa9102092cb6ccc3c1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:39.714924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99b01856a4384dfa9102092cb6ccc3c1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:39.729085 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1387/2767] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:31:39 2022] DELETE /identity/v3/users/99b01856a4384dfa9102092cb6ccc3c1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:31:39.736819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d04d52-9976-4a95-b965-ec1ae9b8197f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:39.738151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d04d52-9976-4a95-b965-ec1ae9b8197f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:39.738151 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d04d52-9976-4a95-b965-ec1ae9b8197f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:39.756027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d2d04d52-9976-4a95-b965-ec1ae9b8197f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:39.757153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d2d04d52-9976-4a95-b965-ec1ae9b8197f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:39.829913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1381/2768] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:39.840243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:39.841701 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:39.910573 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffaebaa-a82d-4537-893b-dbe1fb1d063d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:39.916797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f341fc-e6fe-44db-a727-15240a8cd211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:39.917155 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f341fc-e6fe-44db-a727-15240a8cd211 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:39.918457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f341fc-e6fe-44db-a727-15240a8cd211 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:39.918457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f341fc-e6fe-44db-a727-15240a8cd211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:39.929150 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f341fc-e6fe-44db-a727-15240a8cd211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:39.930617 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f341fc-e6fe-44db-a727-15240a8cd211 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:39.941754 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1388/2769] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:40.237379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d04d52-9976-4a95-b965-ec1ae9b8197f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:40.239798 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d2d04d52-9976-4a95-b965-ec1ae9b8197f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:40.257159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d04d52-9976-4a95-b965-ec1ae9b8197f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:40.262384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:40.263385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:40.263669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:40.263951 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] PATH_INFO: `/v3/projects/31e992228e2640d78f4ef6b46209be8b` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:40.265465 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e992228e2640d78f4ef6b46209be8b)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:40.267433 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:40.320892 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e992228e2640d78f4ef6b46209be8b', 'request_id': 'req-28393ad9-1e08-4361-bc7c-0cced15c8959'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:40.328831 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e992228e2640d78f4ef6b46209be8b', 'request_id': 'req-28393ad9-1e08-4361-bc7c-0cced15c8959'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:40.336106 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e992228e2640d78f4ef6b46209be8b', 'request_id': 'req-28393ad9-1e08-4361-bc7c-0cced15c8959'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:40.347364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] The token cache is being invalidate because project 31e992228e2640d78f4ef6b46209be8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:31:40.347647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:40.348406 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-28393ad9-1e08-4361-bc7c-0cced15c8959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:31:40.350022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1382/2770] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:31:40 2022] DELETE /identity/v3/projects/31e992228e2640d78f4ef6b46209be8b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:31:48.564580 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f341fc-e6fe-44db-a727-15240a8cd211 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:31:48.577258 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f341fc-e6fe-44db-a727-15240a8cd211 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:31:48.635621 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:31:48.636270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:48.636556 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:48.636824 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:48.638833 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:48.675761 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:48.709393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:31:48.710850 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:31:48.726835 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1389/2771] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:31:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:31:50.344110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:31:53.655362 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639f88f-3c97-4c94-af6b-4d6727978159 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:31:53.656034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639f88f-3c97-4c94-af6b-4d6727978159 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:31:53.656034 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639f88f-3c97-4c94-af6b-4d6727978159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:31:53.675615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5639f88f-3c97-4c94-af6b-4d6727978159 None None] MFA Rules not processed for user `642ef5f213254d818de4888cca274311`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:31:53.676925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5639f88f-3c97-4c94-af6b-4d6727978159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:31:53.734512 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1383/2772] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:31:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:31:58.716588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:03.727594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=25 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:40.564437 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:40.566339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:40.578303 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af437236-93db-47ab-8500-35152933dd1e service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:40.583196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:40.583797 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:40.584077 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:40.584347 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:40.586420 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:40.670809 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:40.673121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:40.685942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1390/2773] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:40.955791 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639f88f-3c97-4c94-af6b-4d6727978159 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:40.966789 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639f88f-3c97-4c94-af6b-4d6727978159 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:40.971209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:40.971869 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:40.972046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:40.972299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:40.974276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:40.985598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:40.987086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:40.996967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1384/2774] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:41.106372 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:41.118438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250c39f-1497-43c2-8032-787ebdfb36d8 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:41.123046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853393b-754d-499a-96a2-40e3062a654e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:41.123555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853393b-754d-499a-96a2-40e3062a654e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:41.123836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853393b-754d-499a-96a2-40e3062a654e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:41.124109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853393b-754d-499a-96a2-40e3062a654e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:41.126682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9853393b-754d-499a-96a2-40e3062a654e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:41.140767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853393b-754d-499a-96a2-40e3062a654e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:41.142279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853393b-754d-499a-96a2-40e3062a654e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:41.154493 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1391/2775] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:42.197018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:32:42.198654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:42.210826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ada75-cf8a-4936-a5ab-2c2fbed51158 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:32:42.215029 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:32:42.215695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:32:42.215978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:32:42.216277 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:32:42.218351 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:32:42.233015 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:32:42.234629 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:32:42.247570 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1385/2776] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:32:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:32:51.148629 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=40 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:32:52.243247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=12 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:04.875011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502f2a-04c2-4aec-94da-7bbe0fcc890d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:04.875540 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502f2a-04c2-4aec-94da-7bbe0fcc890d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:04.875638 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3502f2a-04c2-4aec-94da-7bbe0fcc890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:04.893152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e3502f2a-04c2-4aec-94da-7bbe0fcc890d None None] MFA Rules not processed for user `642ef5f213254d818de4888cca274311`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:33:04.894392 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3502f2a-04c2-4aec-94da-7bbe0fcc890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:04.901295 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1392/2777] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:33:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:33:05.244174 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:05.255712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0251f2e-90d0-4022-83b8-1780920915c8 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:05.259693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:05.260329 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:05.260607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:05.260884 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:05.262784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:05.274163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:05.275628 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:05.284104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1386/2778] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:33:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:33:05.621478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3502f2a-04c2-4aec-94da-7bbe0fcc890d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:05.623169 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3502f2a-04c2-4aec-94da-7bbe0fcc890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:05.702512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3502f2a-04c2-4aec-94da-7bbe0fcc890d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:05.708488 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:05.709144 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:05.709458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:05.709730 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:05.712202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:05.724680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:05.726512 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:05.737282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1393/2779] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:33:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:33:07.559840 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:07.569645 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e9d8a-4026-481d-b6f0-c31985f26d2d service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:07.573596 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:07.574265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:07.574552 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:07.574821 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:07.576733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:07.588459 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:07.590146 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:07.600077 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1387/2780] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:33:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:33:08.338825 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:08.341753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:08.359443 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828cff72-6bb1-427d-bad2-7eb100e7bcc9 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:08.366338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:08.367377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:08.367785 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:08.368126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:08.370723 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:08.389668 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:08.391346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:08.405336 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1394/2781] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:33:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:33:16.535061 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:16.536785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:16.554129 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bf1ac0-a9df-4f48-a453-5848a92beb2c service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:16.560156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:16.560818 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:16.561102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:16.561488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:16.563553 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:16.579159 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:16.581213 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:16.592909 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1388/2782] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:33:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:33:18.398880 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=41 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:26.587947 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=18 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:46.771971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:46.784389 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c09a522-ae26-419e-abb9-caccaa8bb1ab service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:46.790019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:46.790683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:46.790976 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:46.791251 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:46.793116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:46.872860 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:46.874331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:46.885575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1395/2783] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:33:53.977548 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:53.980215 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:53.995885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73141f-9b8f-4712-807b-10c7f895361e service nova] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.001018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.001984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.002268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.002542 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] PATH_INFO: `/v3/users/fbbafeb56a4d4fd282275e4198ede783` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.003896 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbbafeb56a4d4fd282275e4198ede783)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.005819 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:54.059510 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbbafeb56a4d4fd282275e4198ede783'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.079621 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbbafeb56a4d4fd282275e4198ede783'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.080395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbbafeb56a4d4fd282275e4198ede783'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.088112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbbafeb56a4d4fd282275e4198ede783'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.097242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbbafeb56a4d4fd282275e4198ede783'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.104656 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1389/2784] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:33:53 2022] DELETE /identity/v3/users/fbbafeb56a4d4fd282275e4198ede783 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:33:54.297648 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:54.308400 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf757791-ea7f-4f66-af2d-c6dad5956598 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.381548 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.382495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.382762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.383042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] PATH_INFO: `/v3/projects/bd1488226a424c6e9e7f94f31b020e4f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.384334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1488226a424c6e9e7f94f31b020e4f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.386290 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:54.440208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1488226a424c6e9e7f94f31b020e4f', 'request_id': 'req-87fbe47d-a152-4c53-bb1f-0d3e3644983c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.445928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1488226a424c6e9e7f94f31b020e4f', 'request_id': 'req-87fbe47d-a152-4c53-bb1f-0d3e3644983c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.454265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1488226a424c6e9e7f94f31b020e4f', 'request_id': 'req-87fbe47d-a152-4c53-bb1f-0d3e3644983c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.476685 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] The token cache is being invalidate because project bd1488226a424c6e9e7f94f31b020e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:33:54.477162 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.478213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.480575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1396/2785] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:33:54 2022] DELETE /identity/v3/projects/bd1488226a424c6e9e7f94f31b020e4f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:33:54.486232 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:54.488527 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:54.503018 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9c6a5-e289-4cd4-a190-3da69cd8b71a admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.572599 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.573440 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.573727 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.574001 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] PATH_INFO: `/v3/users/f1006a405d054de7bbc43c256561a151` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.575234 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1006a405d054de7bbc43c256561a151)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.579185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:54.633853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1006a405d054de7bbc43c256561a151'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.646999 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1006a405d054de7bbc43c256561a151'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.647959 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1006a405d054de7bbc43c256561a151'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.658701 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1006a405d054de7bbc43c256561a151'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.660661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:54.668184 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-8c3e0d6a-6b8d-4ae1-b1df-631beda272fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1006a405d054de7bbc43c256561a151'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:54.675826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1390/2786] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:33:54 2022] DELETE /identity/v3/users/f1006a405d054de7bbc43c256561a151 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:33:54.879683 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:33:54.882193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:33:54.901577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbe47d-a152-4c53-bb1f-0d3e3644983c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:33:54.930110 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:33:54.970355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:33:54.971232 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:33:54.971508 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:33:54.971779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] PATH_INFO: `/v3/projects/6c0384215d0547d4b0e2a2228f06c84d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:33:54.973006 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0384215d0547d4b0e2a2228f06c84d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:33:54.974954 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:33:55.032585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0384215d0547d4b0e2a2228f06c84d', 'request_id': 'req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:55.039518 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0384215d0547d4b0e2a2228f06c84d', 'request_id': 'req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:55.048116 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0384215d0547d4b0e2a2228f06c84d', 'request_id': 'req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:55.061346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] The token cache is being invalidate because project 6c0384215d0547d4b0e2a2228f06c84d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:33:55.061804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:55.063377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:33:55.065248 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1397/2787] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:33:54 2022] DELETE /identity/v3/projects/6c0384215d0547d4b0e2a2228f06c84d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:04.676693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:05.057457 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:14.519027 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2cbc0-f4da-447d-b1a2-d81786b8b878 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:14.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2cbc0-f4da-447d-b1a2-d81786b8b878 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:14.519650 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2cbc0-f4da-447d-b1a2-d81786b8b878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:14.542099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-7fb2cbc0-f4da-447d-b1a2-d81786b8b878 None None] MFA Rules not processed for user `642ef5f213254d818de4888cca274311`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:34:14.543312 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7fb2cbc0-f4da-447d-b1a2-d81786b8b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:14.605899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1391/2788] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:34:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:34:24.598317 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=25 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:24.841384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:24.843539 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:24.975588 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9bac9-463b-4023-886a-d1bd1bd6b47c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:24.982640 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3241452a88a40da9c9b8fa79c4fc9c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:24.984223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:24.984223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:24.984524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:24.987201 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:25.005344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:25.012921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:25.054425 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1398/2789] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:27.339033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2cbc0-f4da-447d-b1a2-d81786b8b878 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:27.340561 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-7fb2cbc0-f4da-447d-b1a2-d81786b8b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:27.413781 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2cbc0-f4da-447d-b1a2-d81786b8b878 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:27.420208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:27.421264 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:27.422093 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:27.422455 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:27.426088 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:27.525836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:27.527310 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:27.545535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1392/2790] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:34:29.573792 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:29.576804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:29.596459 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b471a168-9894-484f-ba5f-bc54dfc8574b service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:29.603758 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:29.604811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:29.605527 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:29.606003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] PATH_INFO: `/v3/users/6cd1d0ce941845c3a28e2cd5fc77c844` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:29.609731 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cd1d0ce941845c3a28e2cd5fc77c844)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:29.612528 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:29.702264 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cd1d0ce941845c3a28e2cd5fc77c844'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:29.703423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cd1d0ce941845c3a28e2cd5fc77c844'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:29.704321 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cd1d0ce941845c3a28e2cd5fc77c844'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:29.724244 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cd1d0ce941845c3a28e2cd5fc77c844'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:29.740848 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cd1d0ce941845c3a28e2cd5fc77c844'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:29.757158 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1399/2791] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:29 2022] DELETE /identity/v3/users/6cd1d0ce941845c3a28e2cd5fc77c844 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:30.057648 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:30.071496 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a55928-3669-4cd8-897c-670421f29b68 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:30.164163 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:30.165335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:30.165622 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:30.165902 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] PATH_INFO: `/v3/projects/50aeadccaa9348fd9af173f94291bf6f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:30.167266 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50aeadccaa9348fd9af173f94291bf6f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:30.169386 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:30.241164 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50aeadccaa9348fd9af173f94291bf6f', 'request_id': 'req-95fb2d05-581e-4990-b297-21f89f6dc850'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.246164 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:30.262466 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50aeadccaa9348fd9af173f94291bf6f', 'request_id': 'req-95fb2d05-581e-4990-b297-21f89f6dc850'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.282198 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50aeadccaa9348fd9af173f94291bf6f', 'request_id': 'req-95fb2d05-581e-4990-b297-21f89f6dc850'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.307588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] The token cache is being invalidate because project 50aeadccaa9348fd9af173f94291bf6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:30.308070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.309400 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.312371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1393/2792] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:30 2022] DELETE /identity/v3/projects/50aeadccaa9348fd9af173f94291bf6f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:30.316743 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:30.318484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:30.343745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1940b0-f0e4-444d-8f07-15185f1d713b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:30.416624 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:30.424316 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:30.426134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:30.426134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:30.426134 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] PATH_INFO: `/v3/users/ff1b16ee76d34b72b236b28dfbd94fb8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:30.428183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff1b16ee76d34b72b236b28dfbd94fb8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:30.430729 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:30.522759 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff1b16ee76d34b72b236b28dfbd94fb8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.523788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff1b16ee76d34b72b236b28dfbd94fb8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.524609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff1b16ee76d34b72b236b28dfbd94fb8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.545237 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff1b16ee76d34b72b236b28dfbd94fb8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.564275 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff1b16ee76d34b72b236b28dfbd94fb8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.584893 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1400/2793] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:34:30 2022] DELETE /identity/v3/users/ff1b16ee76d34b72b236b28dfbd94fb8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:34:30.808247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:34:30.810319 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:34:30.833367 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb2d05-581e-4990-b297-21f89f6dc850 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:34:30.933434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:34:30.934462 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:34:30.934758 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:34:30.935046 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] PATH_INFO: `/v3/projects/8612326dfc09414caca0b216d53b9f13` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:34:30.936365 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8612326dfc09414caca0b216d53b9f13)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:34:30.938320 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:34:30.990703 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8612326dfc09414caca0b216d53b9f13', 'request_id': 'req-3ef8dfaa-50a4-4020-80c7-af426941aa26'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:30.999568 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8612326dfc09414caca0b216d53b9f13', 'request_id': 'req-3ef8dfaa-50a4-4020-80c7-af426941aa26'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:31.007005 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8612326dfc09414caca0b216d53b9f13', 'request_id': 'req-3ef8dfaa-50a4-4020-80c7-af426941aa26'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:31.017895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] The token cache is being invalidate because project 8612326dfc09414caca0b216d53b9f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:34:31.018162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:31.018919 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:34:31.020449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1394/2794] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:34:30 2022] DELETE /identity/v3/projects/8612326dfc09414caca0b216d53b9f13 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:34:40.577151 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:34:41.014525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=10,keystone:SELECT=57,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:26.153712 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.155526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.252412 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5e19cd-7add-4c72-9dce-ec4116ba64e1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.314333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.315240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.315524 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.315803 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] PATH_INFO: `/v3/users/642ef5f213254d818de4888cca274311` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.317082 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=642ef5f213254d818de4888cca274311)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.319109 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.371868 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642ef5f213254d818de4888cca274311'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.372784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642ef5f213254d818de4888cca274311'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.373535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642ef5f213254d818de4888cca274311'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.386521 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642ef5f213254d818de4888cca274311'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.395851 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-cd6482b1-74aa-4f79-b1ad-acce9f313ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642ef5f213254d818de4888cca274311'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.410333 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1401/2795] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:26 2022] DELETE /identity/v3/users/642ef5f213254d818de4888cca274311 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:26.638255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:26.639860 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:26.739214 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef8dfaa-50a4-4020-80c7-af426941aa26 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:26.803891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:26.805069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.805523 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.805872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] PATH_INFO: `/v3/projects/7c56d4c430c44df49f48fe0b49ebcbd2` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.807691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c56d4c430c44df49f48fe0b49ebcbd2)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:26.811290 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:26.867441 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c56d4c430c44df49f48fe0b49ebcbd2', 'request_id': 'req-65925502-2fa1-42b5-a501-0889287e7f32'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.877316 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c56d4c430c44df49f48fe0b49ebcbd2', 'request_id': 'req-65925502-2fa1-42b5-a501-0889287e7f32'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.884765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c56d4c430c44df49f48fe0b49ebcbd2', 'request_id': 'req-65925502-2fa1-42b5-a501-0889287e7f32'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.902438 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] The token cache is being invalidate because project 7c56d4c430c44df49f48fe0b49ebcbd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:26.902733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.904325 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-65925502-2fa1-42b5-a501-0889287e7f32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:26.905291 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1395/2796] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:26 2022] DELETE /identity/v3/projects/7c56d4c430c44df49f48fe0b49ebcbd2 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:26.911020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1240-7e0d-4662-85a7-044ee1ccdcc9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:26.911364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1240-7e0d-4662-85a7-044ee1ccdcc9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:26.911567 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701b1240-7e0d-4662-85a7-044ee1ccdcc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:26.936943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:26.981213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-701b1240-7e0d-4662-85a7-044ee1ccdcc9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:26.983218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-701b1240-7e0d-4662-85a7-044ee1ccdcc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.048353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1402/2797] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:27.056364 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d0183-a5a8-4aa0-a73d-04d6523014c3 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.056655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d0183-a5a8-4aa0-a73d-04d6523014c3 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.056824 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d0183-a5a8-4aa0-a73d-04d6523014c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.077671 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-3c0d0183-a5a8-4aa0-a73d-04d6523014c3 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:27.079028 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3c0d0183-a5a8-4aa0-a73d-04d6523014c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.092044 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1396/2798] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:27.096431 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b1240-7e0d-4662-85a7-044ee1ccdcc9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.106813 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701b1240-7e0d-4662-85a7-044ee1ccdcc9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.112946 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.113677 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.113950 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.114218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.114962 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:27.115268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.116903 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.130112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1403/2799] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:27.134752 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d0183-a5a8-4aa0-a73d-04d6523014c3 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.145135 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d0183-a5a8-4aa0-a73d-04d6523014c3 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.150422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a436ef7-02b6-4e33-ad2e-ffa3da631a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.151102 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436ef7-02b6-4e33-ad2e-ffa3da631a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.151383 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436ef7-02b6-4e33-ad2e-ffa3da631a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.151657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a436ef7-02b6-4e33-ad2e-ffa3da631a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.152360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a436ef7-02b6-4e33-ad2e-ffa3da631a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.154458 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a436ef7-02b6-4e33-ad2e-ffa3da631a72 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.188063 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1397/2800] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:27 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:27.189384 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.200609 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b80f5-4c9d-4166-b48e-a10cc69cd7d5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.266034 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224728e4-d5a9-4f20-a307-5cc6e071e4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.266736 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224728e4-d5a9-4f20-a307-5cc6e071e4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.267020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224728e4-d5a9-4f20-a307-5cc6e071e4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.267318 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224728e4-d5a9-4f20-a307-5cc6e071e4f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.268019 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224728e4-d5a9-4f20-a307-5cc6e071e4f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.270009 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224728e4-d5a9-4f20-a307-5cc6e071e4f5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.296060 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1404/2801] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:27 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:27.300823 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a436ef7-02b6-4e33-ad2e-ffa3da631a72 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.320255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a436ef7-02b6-4e33-ad2e-ffa3da631a72 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.326784 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6e3ed-1ad4-4d46-b64f-c1fd81fab671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.327765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6e3ed-1ad4-4d46-b64f-c1fd81fab671 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.328152 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6e3ed-1ad4-4d46-b64f-c1fd81fab671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.329042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6e3ed-1ad4-4d46-b64f-c1fd81fab671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.329249 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6e3ed-1ad4-4d46-b64f-c1fd81fab671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.330984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a6e3ed-1ad4-4d46-b64f-c1fd81fab671 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.333520 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:27.344737 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1398/2802] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:27.348906 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224728e4-d5a9-4f20-a307-5cc6e071e4f5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.360268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224728e4-d5a9-4f20-a307-5cc6e071e4f5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.365637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c260e507-b5ba-4ef8-9483-8bb591d1647d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.366331 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260e507-b5ba-4ef8-9483-8bb591d1647d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.366614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260e507-b5ba-4ef8-9483-8bb591d1647d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.366891 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c260e507-b5ba-4ef8-9483-8bb591d1647d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.367613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260e507-b5ba-4ef8-9483-8bb591d1647d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.369255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c260e507-b5ba-4ef8-9483-8bb591d1647d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.380338 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1405/2803] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:27.385072 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6e3ed-1ad4-4d46-b64f-c1fd81fab671 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.398254 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6e3ed-1ad4-4d46-b64f-c1fd81fab671 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.403663 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.404573 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.404854 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.405126 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] PATH_INFO: `/v3/projects/4ec8344b44bd4d1d88a615294760633e/users/776cc63d4ab0434da0d76d0148f63d32/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.421522 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec8344b44bd4d1d88a615294760633e, user_id=776cc63d4ab0434da0d76d0148f63d32, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.424344 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.441606 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1399/2804] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:27 2022] PUT /identity/v3/projects/4ec8344b44bd4d1d88a615294760633e/users/776cc63d4ab0434da0d76d0148f63d32/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:27.453346 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38d51d-5056-414a-8d24-68d8f29b457c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.453622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38d51d-5056-414a-8d24-68d8f29b457c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.453924 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38d51d-5056-414a-8d24-68d8f29b457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.484229 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8c38d51d-5056-414a-8d24-68d8f29b457c None None] MFA Rules not processed for user `776cc63d4ab0434da0d76d0148f63d32`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:27.485486 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8c38d51d-5056-414a-8d24-68d8f29b457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.529397 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:27.561255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1406/2805] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:27.572688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:27.574709 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:27.657166 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d29c5ad-8f4e-4cf1-b17e-824a9a9bc64e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:27.661384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69268e-9202-45d3-b364-75a718ae9337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:27.662040 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69268e-9202-45d3-b364-75a718ae9337 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:27.662328 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69268e-9202-45d3-b364-75a718ae9337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:27.662612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69268e-9202-45d3-b364-75a718ae9337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:27.672866 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69268e-9202-45d3-b364-75a718ae9337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:27.674372 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69268e-9202-45d3-b364-75a718ae9337 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:27.684099 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1400/2806] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:36.360452 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38d51d-5056-414a-8d24-68d8f29b457c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:36.437041 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38d51d-5056-414a-8d24-68d8f29b457c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:36.442478 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:36.443379 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:36.443928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:36.444209 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] PATH_INFO: `/v3/users/776cc63d4ab0434da0d76d0148f63d32` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:36.445965 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=776cc63d4ab0434da0d76d0148f63d32)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:36.447939 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:36.518344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '776cc63d4ab0434da0d76d0148f63d32'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:36.519023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '776cc63d4ab0434da0d76d0148f63d32'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:36.519641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '776cc63d4ab0434da0d76d0148f63d32'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:36.534877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '776cc63d4ab0434da0d76d0148f63d32'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:36.544943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '776cc63d4ab0434da0d76d0148f63d32'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:36.566538 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1407/2807] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:36 2022] DELETE /identity/v3/users/776cc63d4ab0434da0d76d0148f63d32 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:36.572602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80813a02-af72-4826-a048-24a5a84e8771 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:36.574108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80813a02-af72-4826-a048-24a5a84e8771 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:36.574108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80813a02-af72-4826-a048-24a5a84e8771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:36.591945 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-80813a02-af72-4826-a048-24a5a84e8771 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:36.593216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-80813a02-af72-4826-a048-24a5a84e8771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:36.663571 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1401/2808] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:36.674363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:36.676037 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:36.768666 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:36.780042 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed40cd9d-0c7f-4e2f-b51c-655be78b7495 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:36.786980 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fffd7b-ee30-481b-9a80-8e676cb09a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:36.787687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fffd7b-ee30-481b-9a80-8e676cb09a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:36.787979 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fffd7b-ee30-481b-9a80-8e676cb09a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:36.788289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fffd7b-ee30-481b-9a80-8e676cb09a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:36.799921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fffd7b-ee30-481b-9a80-8e676cb09a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:36.801442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fffd7b-ee30-481b-9a80-8e676cb09a5e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:36.819784 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1408/2809] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:37.119423 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80813a02-af72-4826-a048-24a5a84e8771 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.120864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-80813a02-af72-4826-a048-24a5a84e8771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:37.139660 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80813a02-af72-4826-a048-24a5a84e8771 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.146153 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.146942 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.147226 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.147579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] PATH_INFO: `/v3/projects/4ec8344b44bd4d1d88a615294760633e` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.148955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec8344b44bd4d1d88a615294760633e)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.151405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:37.208765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec8344b44bd4d1d88a615294760633e', 'request_id': 'req-bf14643d-8366-4e71-90bd-07e53fb028d4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:37.217668 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec8344b44bd4d1d88a615294760633e', 'request_id': 'req-bf14643d-8366-4e71-90bd-07e53fb028d4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:37.218939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:37.225963 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec8344b44bd4d1d88a615294760633e', 'request_id': 'req-bf14643d-8366-4e71-90bd-07e53fb028d4'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:37.239144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] The token cache is being invalidate because project 4ec8344b44bd4d1d88a615294760633e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:37.239437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:37.240091 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-bf14643d-8366-4e71-90bd-07e53fb028d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:37.241816 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1402/2810] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:37 2022] DELETE /identity/v3/projects/4ec8344b44bd4d1d88a615294760633e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:37.247128 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55362-1836-4ccd-9cf5-f537d03e4a4c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.247906 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55362-1836-4ccd-9cf5-f537d03e4a4c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.248195 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55362-1836-4ccd-9cf5-f537d03e4a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.267327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-30f55362-1836-4ccd-9cf5-f537d03e4a4c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:37.268579 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-30f55362-1836-4ccd-9cf5-f537d03e4a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:37.336775 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1409/2811] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:37.345989 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c908aa6-fc27-4ca6-bd4f-554d508fc04c None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.346394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c908aa6-fc27-4ca6-bd4f-554d508fc04c None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.346767 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c908aa6-fc27-4ca6-bd4f-554d508fc04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.367980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-5c908aa6-fc27-4ca6-bd4f-554d508fc04c None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:37.369208 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-5c908aa6-fc27-4ca6-bd4f-554d508fc04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:37.378539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1403/2812] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:37.383435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f55362-1836-4ccd-9cf5-f537d03e4a4c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.394179 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f55362-1836-4ccd-9cf5-f537d03e4a4c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.399582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.400273 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.400555 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.400832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.401611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:37.401921 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.403663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:37.416240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1410/2813] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:37.421103 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c908aa6-fc27-4ca6-bd4f-554d508fc04c None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.433960 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c908aa6-fc27-4ca6-bd4f-554d508fc04c None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.441711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5603ea-0f75-4663-902f-d84341903b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.442385 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5603ea-0f75-4663-902f-d84341903b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.442730 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5603ea-0f75-4663-902f-d84341903b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.442979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5603ea-0f75-4663-902f-d84341903b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.444052 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5603ea-0f75-4663-902f-d84341903b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.446377 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5603ea-0f75-4663-902f-d84341903b61 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:37.489776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1404/2814] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:37 2022] POST /identity/v3/projects => generated 363 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:37.490912 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.501716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d54af71-e7e0-46a2-a03f-a6775eb937c8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.563090 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbf3ba-749e-4a54-a294-3dc4679438c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.563769 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf3ba-749e-4a54-a294-3dc4679438c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.564051 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf3ba-749e-4a54-a294-3dc4679438c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.564339 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbbf3ba-749e-4a54-a294-3dc4679438c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.565027 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbf3ba-749e-4a54-a294-3dc4679438c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.566987 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbbf3ba-749e-4a54-a294-3dc4679438c1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:37.590441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1411/2815] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:37 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 17:35:37.595382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5603ea-0f75-4663-902f-d84341903b61 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.616655 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5603ea-0f75-4663-902f-d84341903b61 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.624615 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe74f5-b716-42e9-a3c9-585e80dfead5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.625322 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe74f5-b716-42e9-a3c9-585e80dfead5 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.625588 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe74f5-b716-42e9-a3c9-585e80dfead5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.625859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe74f5-b716-42e9-a3c9-585e80dfead5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.626594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe74f5-b716-42e9-a3c9-585e80dfead5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.628182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe74f5-b716-42e9-a3c9-585e80dfead5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:37.639868 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1405/2816] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:37.644172 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbf3ba-749e-4a54-a294-3dc4679438c1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.655551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbbf3ba-749e-4a54-a294-3dc4679438c1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.662745 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b92223-350f-49ee-8f8f-9ea1e29bae3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.663832 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b92223-350f-49ee-8f8f-9ea1e29bae3c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.664310 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b92223-350f-49ee-8f8f-9ea1e29bae3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.664647 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b92223-350f-49ee-8f8f-9ea1e29bae3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.666125 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b92223-350f-49ee-8f8f-9ea1e29bae3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.672467 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b92223-350f-49ee-8f8f-9ea1e29bae3c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:37.686639 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1412/2817] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:37.691543 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe74f5-b716-42e9-a3c9-585e80dfead5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.702634 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe74f5-b716-42e9-a3c9-585e80dfead5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.707711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3b068-e898-4b60-81cc-4abe08d9462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.708618 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3b068-e898-4b60-81cc-4abe08d9462d admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.708899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3b068-e898-4b60-81cc-4abe08d9462d admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.709205 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3b068-e898-4b60-81cc-4abe08d9462d admin admin] PATH_INFO: `/v3/projects/7e97ce5459c14531b899453821c81038/users/00f635718b9c4dadb0c08b93a037ff4f/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.729646 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3b068-e898-4b60-81cc-4abe08d9462d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e97ce5459c14531b899453821c81038, user_id=00f635718b9c4dadb0c08b93a037ff4f, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.732738 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3b068-e898-4b60-81cc-4abe08d9462d admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:37.748303 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1406/2818] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:37 2022] PUT /identity/v3/projects/7e97ce5459c14531b899453821c81038/users/00f635718b9c4dadb0c08b93a037ff4f/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:37.760344 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c729d-e413-4ef1-af16-614e5c40cf20 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.760776 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c729d-e413-4ef1-af16-614e5c40cf20 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.761070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c729d-e413-4ef1-af16-614e5c40cf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.790471 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-517c729d-e413-4ef1-af16-614e5c40cf20 None None] MFA Rules not processed for user `00f635718b9c4dadb0c08b93a037ff4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:37.791698 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-517c729d-e413-4ef1-af16-614e5c40cf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:37.839988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:37.878905 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1413/2819] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:37 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:37.884772 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3b068-e898-4b60-81cc-4abe08d9462d admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:37.960335 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3b068-e898-4b60-81cc-4abe08d9462d admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:37.965930 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b044f-2dbe-4194-a67c-4dab126d05a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:37.966586 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b044f-2dbe-4194-a67c-4dab126d05a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:37.966879 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b044f-2dbe-4194-a67c-4dab126d05a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:37.967182 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b044f-2dbe-4194-a67c-4dab126d05a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:37.967864 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b044f-2dbe-4194-a67c-4dab126d05a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:37.969811 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b044f-2dbe-4194-a67c-4dab126d05a8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:38.004060 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1407/2820] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:37 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:38.008858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c729d-e413-4ef1-af16-614e5c40cf20 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:38.021126 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c729d-e413-4ef1-af16-614e5c40cf20 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:38.082972 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33aa1f-6888-4aec-b4f2-1fa5aa38a783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:38.083815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33aa1f-6888-4aec-b4f2-1fa5aa38a783 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:38.084106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33aa1f-6888-4aec-b4f2-1fa5aa38a783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:38.084409 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33aa1f-6888-4aec-b4f2-1fa5aa38a783 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:38.085124 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33aa1f-6888-4aec-b4f2-1fa5aa38a783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:38.087070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33aa1f-6888-4aec-b4f2-1fa5aa38a783 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:38.108193 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1414/2821] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:38 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:38.112654 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b044f-2dbe-4194-a67c-4dab126d05a8 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:38.124087 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b044f-2dbe-4194-a67c-4dab126d05a8 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:38.129578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa424e-86c6-45de-a752-961dadb4f263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:38.130262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa424e-86c6-45de-a752-961dadb4f263 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:38.130551 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa424e-86c6-45de-a752-961dadb4f263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:38.130833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa424e-86c6-45de-a752-961dadb4f263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:38.131653 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa424e-86c6-45de-a752-961dadb4f263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:38.133309 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa424e-86c6-45de-a752-961dadb4f263 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:38.146760 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1408/2822] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:35:38.151254 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33aa1f-6888-4aec-b4f2-1fa5aa38a783 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:38.162378 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33aa1f-6888-4aec-b4f2-1fa5aa38a783 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:38.168622 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c276e95-5e5b-4037-b5c8-699a625c9521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:38.170002 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276e95-5e5b-4037-b5c8-699a625c9521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:38.170285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276e95-5e5b-4037-b5c8-699a625c9521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:38.170644 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c276e95-5e5b-4037-b5c8-699a625c9521 admin admin] PATH_INFO: `/v3/projects/de07fc95d6d34122bd3d1ed4b6d04c7f/users/99b463b7bb694311acaaf1b4a5aa2907/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:38.189753 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c276e95-5e5b-4037-b5c8-699a625c9521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de07fc95d6d34122bd3d1ed4b6d04c7f, user_id=99b463b7bb694311acaaf1b4a5aa2907, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:38.193439 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c276e95-5e5b-4037-b5c8-699a625c9521 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:38.209643 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1415/2823] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:38 2022] PUT /identity/v3/projects/de07fc95d6d34122bd3d1ed4b6d04c7f/users/99b463b7bb694311acaaf1b4a5aa2907/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:38.213637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa424e-86c6-45de-a752-961dadb4f263 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:38.225535 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa424e-86c6-45de-a752-961dadb4f263 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:38.291852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:38.300161 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f1c97-63fd-42d3-9e92-ab8645be32e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:38.301127 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f1c97-63fd-42d3-9e92-ab8645be32e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:38.301616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f1c97-63fd-42d3-9e92-ab8645be32e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:38.302324 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f1c97-63fd-42d3-9e92-ab8645be32e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:38.303428 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f1c97-63fd-42d3-9e92-ab8645be32e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:38.305537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f1c97-63fd-42d3-9e92-ab8645be32e0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:38.316262 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1409/2824] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:38.320608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c276e95-5e5b-4037-b5c8-699a625c9521 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:38.341808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c276e95-5e5b-4037-b5c8-699a625c9521 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:38.346916 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:38.347782 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:38.348073 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:38.348371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] PATH_INFO: `/v3/projects/de07fc95d6d34122bd3d1ed4b6d04c7f/users/99b463b7bb694311acaaf1b4a5aa2907/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:38.350423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de07fc95d6d34122bd3d1ed4b6d04c7f, user_id=99b463b7bb694311acaaf1b4a5aa2907, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:38.361739 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:38.376436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1416/2825] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:38 2022] PUT /identity/v3/projects/de07fc95d6d34122bd3d1ed4b6d04c7f/users/99b463b7bb694311acaaf1b4a5aa2907/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:38.388419 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46926-e737-44a3-8d7a-77209bb91760 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:38.388614 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46926-e737-44a3-8d7a-77209bb91760 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:38.388873 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf46926-e737-44a3-8d7a-77209bb91760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:38.422104 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-caf46926-e737-44a3-8d7a-77209bb91760 None None] MFA Rules not processed for user `99b463b7bb694311acaaf1b4a5aa2907`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:38.423938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-caf46926-e737-44a3-8d7a-77209bb91760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:38.516167 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1410/2826] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:38.528023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:38.530788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:38.597100 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d69389-72c4-44ed-8763-5b9a71cf4e8d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:38.601202 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893dc79-1ea8-4022-aa82-0f3c7e4b2aae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:38.601853 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893dc79-1ea8-4022-aa82-0f3c7e4b2aae service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:38.602194 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893dc79-1ea8-4022-aa82-0f3c7e4b2aae service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:38.602423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893dc79-1ea8-4022-aa82-0f3c7e4b2aae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:38.613553 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893dc79-1ea8-4022-aa82-0f3c7e4b2aae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:38.615004 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893dc79-1ea8-4022-aa82-0f3c7e4b2aae service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:38.625819 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1417/2827] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:38.712287 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf46926-e737-44a3-8d7a-77209bb91760 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:38.723833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf46926-e737-44a3-8d7a-77209bb91760 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:38.727762 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542609c-dd41-422b-8d4a-48f4a2aa0b79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:38.728425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542609c-dd41-422b-8d4a-48f4a2aa0b79 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:38.728711 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542609c-dd41-422b-8d4a-48f4a2aa0b79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:38.729011 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542609c-dd41-422b-8d4a-48f4a2aa0b79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:38.813456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542609c-dd41-422b-8d4a-48f4a2aa0b79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:38.814890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542609c-dd41-422b-8d4a-48f4a2aa0b79 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:38.825212 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1411/2828] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:38 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:48.620577 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:48.820302 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=66 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:49.491627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893dc79-1ea8-4022-aa82-0f3c7e4b2aae service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:49.581988 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893dc79-1ea8-4022-aa82-0f3c7e4b2aae service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:49.592375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:49.593815 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:49.594208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:49.594687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] PATH_INFO: `/v3/users/00f635718b9c4dadb0c08b93a037ff4f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:49.596582 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f635718b9c4dadb0c08b93a037ff4f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:49.599719 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:49.664557 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f635718b9c4dadb0c08b93a037ff4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:49.665461 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f635718b9c4dadb0c08b93a037ff4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:49.665841 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f635718b9c4dadb0c08b93a037ff4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:49.679433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f635718b9c4dadb0c08b93a037ff4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:49.688484 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f635718b9c4dadb0c08b93a037ff4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:49.702259 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1418/2829] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:49 2022] DELETE /identity/v3/users/00f635718b9c4dadb0c08b93a037ff4f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:49.708803 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f21690-05e1-48c8-badc-27535afc500d None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:49.709085 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f21690-05e1-48c8-badc-27535afc500d None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:49.709368 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f21690-05e1-48c8-badc-27535afc500d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:49.728425 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-77f21690-05e1-48c8-badc-27535afc500d None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:49.729691 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-77f21690-05e1-48c8-badc-27535afc500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:49.800712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1412/2830] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:49.813106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:49.814708 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:49.887106 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715285c0-0696-4adb-9ac9-a55d2af74433 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:49.891806 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cf69b-7efe-4851-9d1e-d2bba023df61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:49.892479 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cf69b-7efe-4851-9d1e-d2bba023df61 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:49.892764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cf69b-7efe-4851-9d1e-d2bba023df61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:49.893026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cf69b-7efe-4851-9d1e-d2bba023df61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:49.912781 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cf69b-7efe-4851-9d1e-d2bba023df61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:49.914327 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cf69b-7efe-4851-9d1e-d2bba023df61 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:49.926767 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1419/2831] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:50.251409 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f21690-05e1-48c8-badc-27535afc500d None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:50.253732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-77f21690-05e1-48c8-badc-27535afc500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:50.270785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f21690-05e1-48c8-badc-27535afc500d None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:50.275685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:50.276437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:50.276716 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:50.276991 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] PATH_INFO: `/v3/projects/7e97ce5459c14531b899453821c81038` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:50.278492 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e97ce5459c14531b899453821c81038)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:50.280375 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:50.349916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e97ce5459c14531b899453821c81038', 'request_id': 'req-712e9808-2edb-4f6a-97fc-c55b6e251f7e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.366973 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e97ce5459c14531b899453821c81038', 'request_id': 'req-712e9808-2edb-4f6a-97fc-c55b6e251f7e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.383017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e97ce5459c14531b899453821c81038', 'request_id': 'req-712e9808-2edb-4f6a-97fc-c55b6e251f7e'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.396944 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] The token cache is being invalidate because project 7e97ce5459c14531b899453821c81038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:50.397281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.397957 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.399658 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1413/2832] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:50 2022] DELETE /identity/v3/projects/7e97ce5459c14531b899453821c81038 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:50.403839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cf69b-7efe-4851-9d1e-d2bba023df61 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:50.417053 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cf69b-7efe-4851-9d1e-d2bba023df61 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:50.440938 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:50.495203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:50.496606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:50.496774 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:50.496997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] PATH_INFO: `/v3/users/99b463b7bb694311acaaf1b4a5aa2907` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:50.498371 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99b463b7bb694311acaaf1b4a5aa2907)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:50.500356 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:50.555026 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99b463b7bb694311acaaf1b4a5aa2907'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.555696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99b463b7bb694311acaaf1b4a5aa2907'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.556288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99b463b7bb694311acaaf1b4a5aa2907'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.569631 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99b463b7bb694311acaaf1b4a5aa2907'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.579240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f43c04dd-fba3-42b8-a5db-e4df585601f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99b463b7bb694311acaaf1b4a5aa2907'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:50.592952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1420/2833] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:50 2022] DELETE /identity/v3/users/99b463b7bb694311acaaf1b4a5aa2907 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:50.912235 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:50.913379 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:50.929899 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e9808-2edb-4f6a-97fc-c55b6e251f7e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.005437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.006218 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.006500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.006777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] PATH_INFO: `/v3/projects/de07fc95d6d34122bd3d1ed4b6d04c7f` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.008136 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] RBAC: Authorizing `identity:delete_project(project_id=de07fc95d6d34122bd3d1ed4b6d04c7f)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.010130 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.045700 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:51.067162 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de07fc95d6d34122bd3d1ed4b6d04c7f', 'request_id': 'req-ff4a2288-8158-47a8-adc6-f1422a34cfae'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:51.079407 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de07fc95d6d34122bd3d1ed4b6d04c7f', 'request_id': 'req-ff4a2288-8158-47a8-adc6-f1422a34cfae'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:51.086746 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de07fc95d6d34122bd3d1ed4b6d04c7f', 'request_id': 'req-ff4a2288-8158-47a8-adc6-f1422a34cfae'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:51.099877 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] The token cache is being invalidate because project de07fc95d6d34122bd3d1ed4b6d04c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:51.100144 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:51.100852 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ff4a2288-8158-47a8-adc6-f1422a34cfae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:51.102473 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1414/2834] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:50 2022] DELETE /identity/v3/projects/de07fc95d6d34122bd3d1ed4b6d04c7f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:51.107847 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2adfa7-2ce4-4f85-afa3-3e7c2b673994 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.108458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2adfa7-2ce4-4f85-afa3-3e7c2b673994 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.108458 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2adfa7-2ce4-4f85-afa3-3e7c2b673994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.132788 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fe2adfa7-2ce4-4f85-afa3-3e7c2b673994 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:51.134080 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fe2adfa7-2ce4-4f85-afa3-3e7c2b673994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:51.208764 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1421/2835] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:51.217728 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75136-80d8-4f0b-93af-7cfb76e608d4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.218012 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75136-80d8-4f0b-93af-7cfb76e608d4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.218292 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d75136-80d8-4f0b-93af-7cfb76e608d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.239079 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-04d75136-80d8-4f0b-93af-7cfb76e608d4 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:51.240382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-04d75136-80d8-4f0b-93af-7cfb76e608d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:51.250422 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1415/2836] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:51.256298 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2adfa7-2ce4-4f85-afa3-3e7c2b673994 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.267771 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2adfa7-2ce4-4f85-afa3-3e7c2b673994 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.273746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.274595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.274937 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.275208 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.275949 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:51.276270 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.278693 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.294410 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1422/2837] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:51.299201 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d75136-80d8-4f0b-93af-7cfb76e608d4 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.309452 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d75136-80d8-4f0b-93af-7cfb76e608d4 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.315045 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251af408-3323-41aa-8124-a991c10df5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.315747 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251af408-3323-41aa-8124-a991c10df5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.316023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251af408-3323-41aa-8124-a991c10df5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.316299 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251af408-3323-41aa-8124-a991c10df5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.318867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251af408-3323-41aa-8124-a991c10df5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.319722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251af408-3323-41aa-8124-a991c10df5fb admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.352633 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1416/2838] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:51 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:51.357483 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.368615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a20bd-d659-4ce1-8587-1f2fc4ca8d6a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.400093 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:51.442669 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f500726-19cd-4046-be09-f375f0c9d54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.443394 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500726-19cd-4046-be09-f375f0c9d54f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.443680 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500726-19cd-4046-be09-f375f0c9d54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.443961 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f500726-19cd-4046-be09-f375f0c9d54f admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.444682 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f500726-19cd-4046-be09-f375f0c9d54f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.446659 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f500726-19cd-4046-be09-f375f0c9d54f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.473289 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1423/2839] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:51 2022] POST /identity/v3/users => generated 283 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:51.478813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251af408-3323-41aa-8124-a991c10df5fb admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.498669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251af408-3323-41aa-8124-a991c10df5fb admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.506664 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083027df-3c2c-49d2-b90a-a74222b56bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.507674 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083027df-3c2c-49d2-b90a-a74222b56bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.508123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083027df-3c2c-49d2-b90a-a74222b56bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.508620 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083027df-3c2c-49d2-b90a-a74222b56bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.510418 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083027df-3c2c-49d2-b90a-a74222b56bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.513247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083027df-3c2c-49d2-b90a-a74222b56bc0 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.525979 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1417/2840] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:51.532196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f500726-19cd-4046-be09-f375f0c9d54f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.547301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f500726-19cd-4046-be09-f375f0c9d54f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.553932 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dfe2d-0d5a-4a3f-89d1-6162e3a9d03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.554762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfe2d-0d5a-4a3f-89d1-6162e3a9d03e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.554762 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfe2d-0d5a-4a3f-89d1-6162e3a9d03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.555374 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dfe2d-0d5a-4a3f-89d1-6162e3a9d03e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.556103 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dfe2d-0d5a-4a3f-89d1-6162e3a9d03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.557802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dfe2d-0d5a-4a3f-89d1-6162e3a9d03e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.573741 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1424/2841] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:51.580237 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083027df-3c2c-49d2-b90a-a74222b56bc0 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.594177 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083027df-3c2c-49d2-b90a-a74222b56bc0 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.599888 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904685d-fab2-4c54-9453-2788dd6c9416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.601456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904685d-fab2-4c54-9453-2788dd6c9416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.601456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904685d-fab2-4c54-9453-2788dd6c9416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.601456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904685d-fab2-4c54-9453-2788dd6c9416 admin admin] PATH_INFO: `/v3/projects/08c43d6e9cd04d92ab518d3b88a71c77/users/2a237a43a3c04817950b0152b96dd78f/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.618371 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904685d-fab2-4c54-9453-2788dd6c9416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c43d6e9cd04d92ab518d3b88a71c77, user_id=2a237a43a3c04817950b0152b96dd78f, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.621115 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904685d-fab2-4c54-9453-2788dd6c9416 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.638470 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1418/2842] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:51 2022] PUT /identity/v3/projects/08c43d6e9cd04d92ab518d3b88a71c77/users/2a237a43a3c04817950b0152b96dd78f/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:51.650357 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2c9a8-1224-44f8-9d78-874ab4864077 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.650646 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2c9a8-1224-44f8-9d78-874ab4864077 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.650911 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2c9a8-1224-44f8-9d78-874ab4864077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.686606 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-89e2c9a8-1224-44f8-9d78-874ab4864077 None None] MFA Rules not processed for user `2a237a43a3c04817950b0152b96dd78f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:51.687928 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-89e2c9a8-1224-44f8-9d78-874ab4864077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:51.762696 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1425/2843] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:51.767170 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904685d-fab2-4c54-9453-2788dd6c9416 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.843882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904685d-fab2-4c54-9453-2788dd6c9416 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.851113 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b33e40-6bd2-44a1-9036-7adac330f37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.851779 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33e40-6bd2-44a1-9036-7adac330f37f admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.852058 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33e40-6bd2-44a1-9036-7adac330f37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.852334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b33e40-6bd2-44a1-9036-7adac330f37f admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.853062 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b33e40-6bd2-44a1-9036-7adac330f37f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.855280 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b33e40-6bd2-44a1-9036-7adac330f37f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.885449 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1419/2844] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:51 2022] POST /identity/v3/projects => generated 365 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:51.891292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2c9a8-1224-44f8-9d78-874ab4864077 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:51.903049 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2c9a8-1224-44f8-9d78-874ab4864077 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:51.968576 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1817e-dd5e-42fc-8039-dfeffad14cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:51.969323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1817e-dd5e-42fc-8039-dfeffad14cfa admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:51.969611 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1817e-dd5e-42fc-8039-dfeffad14cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:51.969882 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1817e-dd5e-42fc-8039-dfeffad14cfa admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:51.970614 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1817e-dd5e-42fc-8039-dfeffad14cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:51.972953 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1817e-dd5e-42fc-8039-dfeffad14cfa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:51.998442 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1426/2845] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:51 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:52.002686 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b33e40-6bd2-44a1-9036-7adac330f37f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:52.014384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b33e40-6bd2-44a1-9036-7adac330f37f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:52.020156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5baca-8bb1-4cb5-9f5d-17bbd4d73794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:52.020844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5baca-8bb1-4cb5-9f5d-17bbd4d73794 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:52.021124 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5baca-8bb1-4cb5-9f5d-17bbd4d73794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:52.021436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5baca-8bb1-4cb5-9f5d-17bbd4d73794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:52.022171 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5baca-8bb1-4cb5-9f5d-17bbd4d73794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:52.023826 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5baca-8bb1-4cb5-9f5d-17bbd4d73794 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:52.037714 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1420/2846] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:52.042593 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1817e-dd5e-42fc-8039-dfeffad14cfa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:52.055423 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1817e-dd5e-42fc-8039-dfeffad14cfa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:52.067255 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674abb25-3fc3-4399-8fa1-d31c8a948e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:52.070641 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674abb25-3fc3-4399-8fa1-d31c8a948e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:52.070923 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674abb25-3fc3-4399-8fa1-d31c8a948e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:52.071200 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674abb25-3fc3-4399-8fa1-d31c8a948e78 admin admin] PATH_INFO: `/v3/projects/b6968ba8436f4bbd87122f8c12c56df7/users/2e30295fd1ca4c61a01801e797936b7c/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:52.090432 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674abb25-3fc3-4399-8fa1-d31c8a948e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6968ba8436f4bbd87122f8c12c56df7, user_id=2e30295fd1ca4c61a01801e797936b7c, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:52.093306 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674abb25-3fc3-4399-8fa1-d31c8a948e78 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:52.112013 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1427/2847] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:52 2022] PUT /identity/v3/projects/b6968ba8436f4bbd87122f8c12c56df7/users/2e30295fd1ca4c61a01801e797936b7c/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:52.117636 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5baca-8bb1-4cb5-9f5d-17bbd4d73794 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:52.128731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5baca-8bb1-4cb5-9f5d-17bbd4d73794 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:52.151849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:52.201761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112f0d96-98fc-4714-8ecb-e06e287900c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:52.202437 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f0d96-98fc-4714-8ecb-e06e287900c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:52.202712 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f0d96-98fc-4714-8ecb-e06e287900c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:52.205242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112f0d96-98fc-4714-8ecb-e06e287900c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:52.205242 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f0d96-98fc-4714-8ecb-e06e287900c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:52.205495 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112f0d96-98fc-4714-8ecb-e06e287900c3 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:52.216209 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1421/2848] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 17:35:52.220589 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674abb25-3fc3-4399-8fa1-d31c8a948e78 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:52.226558 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:52.234121 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674abb25-3fc3-4399-8fa1-d31c8a948e78 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:52.241509 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:52.242441 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:52.242713 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:52.242977 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] PATH_INFO: `/v3/projects/b6968ba8436f4bbd87122f8c12c56df7/users/2e30295fd1ca4c61a01801e797936b7c/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:52.245102 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6968ba8436f4bbd87122f8c12c56df7, user_id=2e30295fd1ca4c61a01801e797936b7c, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:52.248634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:52.261444 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1428/2849] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:52 2022] PUT /identity/v3/projects/b6968ba8436f4bbd87122f8c12c56df7/users/2e30295fd1ca4c61a01801e797936b7c/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:35:52.281733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ef6b-77e7-4533-8c6c-cf04c8545428 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:52.281733 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ef6b-77e7-4533-8c6c-cf04c8545428 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:52.284380 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045ef6b-77e7-4533-8c6c-cf04c8545428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:52.314417 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-8045ef6b-77e7-4533-8c6c-cf04c8545428 None None] MFA Rules not processed for user `2e30295fd1ca4c61a01801e797936b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:52.315672 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-8045ef6b-77e7-4533-8c6c-cf04c8545428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:52.396181 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1422/2850] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:52.411382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:52.414107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:52.486808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ff874-dd9c-4cfe-b405-67798b19c712 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:52.490992 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f00c7-96ee-46b3-b123-f55c152fe4cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:52.491663 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f00c7-96ee-46b3-b123-f55c152fe4cc service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:52.491944 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f00c7-96ee-46b3-b123-f55c152fe4cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:52.492212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f00c7-96ee-46b3-b123-f55c152fe4cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:52.503323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f00c7-96ee-46b3-b123-f55c152fe4cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:52.506804 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f00c7-96ee-46b3-b123-f55c152fe4cc service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:52.519535 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1429/2851] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:35:52.608500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045ef6b-77e7-4533-8c6c-cf04c8545428 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:52.623580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045ef6b-77e7-4533-8c6c-cf04c8545428 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:52.628185 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd62220-2d46-4b12-bd9b-692614a5063c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:52.628855 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62220-2d46-4b12-bd9b-692614a5063c service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:52.629134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62220-2d46-4b12-bd9b-692614a5063c service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:52.629450 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd62220-2d46-4b12-bd9b-692614a5063c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:52.696595 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd62220-2d46-4b12-bd9b-692614a5063c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:52.698095 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd62220-2d46-4b12-bd9b-692614a5063c service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:52.709382 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1423/2852] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:55.599610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f00c7-96ee-46b3-b123-f55c152fe4cc service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:55.672413 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f00c7-96ee-46b3-b123-f55c152fe4cc service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:55.678021 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:55.679268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:55.679268 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:55.679526 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] PATH_INFO: `/v3/users/2a237a43a3c04817950b0152b96dd78f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:55.681354 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a237a43a3c04817950b0152b96dd78f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:55.683438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:55.747681 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a237a43a3c04817950b0152b96dd78f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:55.748252 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a237a43a3c04817950b0152b96dd78f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:55.748892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a237a43a3c04817950b0152b96dd78f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:55.761480 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a237a43a3c04817950b0152b96dd78f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:55.770250 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a237a43a3c04817950b0152b96dd78f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:55.784282 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1430/2853] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:55 2022] DELETE /identity/v3/users/2a237a43a3c04817950b0152b96dd78f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:55.789718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664572d5-f4e8-487f-8d1d-170080634f7f None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:55.790394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664572d5-f4e8-487f-8d1d-170080634f7f None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:55.790394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664572d5-f4e8-487f-8d1d-170080634f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:55.810187 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-664572d5-f4e8-487f-8d1d-170080634f7f None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:55.811464 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-664572d5-f4e8-487f-8d1d-170080634f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:55.866261 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=100 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:55.889156 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1424/2854] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:55.900369 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:55.902181 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:55.922245 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:55.990292 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244e7de-f244-4d6b-bbe8-f3347fe6db62 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:55.994382 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f9d27-ea08-427c-b338-b157978f0c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:55.995070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f9d27-ea08-427c-b338-b157978f0c57 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:55.995353 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f9d27-ea08-427c-b338-b157978f0c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:55.995635 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f9d27-ea08-427c-b338-b157978f0c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:56.006154 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f9d27-ea08-427c-b338-b157978f0c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:56.007520 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f9d27-ea08-427c-b338-b157978f0c57 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:56.021096 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1431/2855] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:56.335460 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664572d5-f4e8-487f-8d1d-170080634f7f None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:56.336971 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-664572d5-f4e8-487f-8d1d-170080634f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:56.349591 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664572d5-f4e8-487f-8d1d-170080634f7f None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:56.354897 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:56.355846 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:56.356123 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:56.356489 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] PATH_INFO: `/v3/projects/08c43d6e9cd04d92ab518d3b88a71c77` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:56.358204 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c43d6e9cd04d92ab518d3b88a71c77)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:56.360378 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:56.429363 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c43d6e9cd04d92ab518d3b88a71c77', 'request_id': 'req-3038e578-216f-45cc-8129-a0d83555ddc9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.437680 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c43d6e9cd04d92ab518d3b88a71c77', 'request_id': 'req-3038e578-216f-45cc-8129-a0d83555ddc9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.444889 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c43d6e9cd04d92ab518d3b88a71c77', 'request_id': 'req-3038e578-216f-45cc-8129-a0d83555ddc9'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.457461 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] The token cache is being invalidate because project 08c43d6e9cd04d92ab518d3b88a71c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:56.457750 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.458429 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.460057 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1425/2856] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:56 2022] DELETE /identity/v3/projects/08c43d6e9cd04d92ab518d3b88a71c77 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:56.464183 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f9d27-ea08-427c-b338-b157978f0c57 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:56.476324 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f9d27-ea08-427c-b338-b157978f0c57 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:56.569742 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:56.570802 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:56.571131 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:56.571472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] PATH_INFO: `/v3/users/2e30295fd1ca4c61a01801e797936b7c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:56.573011 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e30295fd1ca4c61a01801e797936b7c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:56.575448 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:56.638120 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e30295fd1ca4c61a01801e797936b7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.638968 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e30295fd1ca4c61a01801e797936b7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.639642 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e30295fd1ca4c61a01801e797936b7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.654805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e30295fd1ca4c61a01801e797936b7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.663464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aa9aa31d-b058-4279-bf9e-526a897c8d3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e30295fd1ca4c61a01801e797936b7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:56.677732 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1432/2857] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:35:56 2022] DELETE /identity/v3/users/2e30295fd1ca4c61a01801e797936b7c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:57.019346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.025295 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:57.044112 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038e578-216f-45cc-8129-a0d83555ddc9 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:57.110616 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:57.111809 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.112086 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.112360 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] PATH_INFO: `/v3/projects/b6968ba8436f4bbd87122f8c12c56df7` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.113833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6968ba8436f4bbd87122f8c12c56df7)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:57.115825 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:57.169353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6968ba8436f4bbd87122f8c12c56df7', 'request_id': 'req-1fc59d22-b559-4de0-9330-0fbcf3308965'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:57.178500 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6968ba8436f4bbd87122f8c12c56df7', 'request_id': 'req-1fc59d22-b559-4de0-9330-0fbcf3308965'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:57.186640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6968ba8436f4bbd87122f8c12c56df7', 'request_id': 'req-1fc59d22-b559-4de0-9330-0fbcf3308965'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:57.195637 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:57.204905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] The token cache is being invalidate because project b6968ba8436f4bbd87122f8c12c56df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:35:57.205265 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:57.206765 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-1fc59d22-b559-4de0-9330-0fbcf3308965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:35:57.210435 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1426/2858] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:35:57 2022] DELETE /identity/v3/projects/b6968ba8436f4bbd87122f8c12c56df7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:57.215768 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735046ae-6eee-4b39-bb5e-bab2c81502d9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.216064 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735046ae-6eee-4b39-bb5e-bab2c81502d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.216377 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735046ae-6eee-4b39-bb5e-bab2c81502d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.236943 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-735046ae-6eee-4b39-bb5e-bab2c81502d9 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:57.238196 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-735046ae-6eee-4b39-bb5e-bab2c81502d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:57.271334 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:57.306101 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1433/2859] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:57.314384 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65045-6756-4218-ba78-7130a6829347 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.314687 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65045-6756-4218-ba78-7130a6829347 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.314949 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d65045-6756-4218-ba78-7130a6829347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.333976 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-89d65045-6756-4218-ba78-7130a6829347 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:57.335143 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-89d65045-6756-4218-ba78-7130a6829347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:57.343867 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1427/2860] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:57.349288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735046ae-6eee-4b39-bb5e-bab2c81502d9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.366139 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735046ae-6eee-4b39-bb5e-bab2c81502d9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:57.370952 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:57.371746 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.372033 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.372313 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.373070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:35:57.373411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:57.375046 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:57.388455 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1434/2861] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:35:57.393984 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d65045-6756-4218-ba78-7130a6829347 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.404745 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d65045-6756-4218-ba78-7130a6829347 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:57.409768 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98323946-c3d7-42f3-a937-204888e95528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:57.410456 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98323946-c3d7-42f3-a937-204888e95528 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.410741 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98323946-c3d7-42f3-a937-204888e95528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.411017 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98323946-c3d7-42f3-a937-204888e95528 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.411718 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98323946-c3d7-42f3-a937-204888e95528 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:57.413661 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98323946-c3d7-42f3-a937-204888e95528 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:57.443842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1428/2862] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:57 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:57.448603 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.460005 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a0f70-193e-4faa-99a2-73746f34b047 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:57.534038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bdbbb-1296-4663-800b-f55f47cce547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:57.534778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bdbbb-1296-4663-800b-f55f47cce547 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.535062 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bdbbb-1296-4663-800b-f55f47cce547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.535494 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bdbbb-1296-4663-800b-f55f47cce547 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.536213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bdbbb-1296-4663-800b-f55f47cce547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:57.538210 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12bdbbb-1296-4663-800b-f55f47cce547 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:57.564288 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1435/2863] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:57 2022] POST /identity/v3/users => generated 301 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:57.567722 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98323946-c3d7-42f3-a937-204888e95528 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.579891 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98323946-c3d7-42f3-a937-204888e95528 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:57.585744 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1a7fd-5487-47d3-a7a9-e0563c63c371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:57.586472 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a7fd-5487-47d3-a7a9-e0563c63c371 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.586693 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a7fd-5487-47d3-a7a9-e0563c63c371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.586965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad1a7fd-5487-47d3-a7a9-e0563c63c371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.587695 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1a7fd-5487-47d3-a7a9-e0563c63c371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:57.589486 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad1a7fd-5487-47d3-a7a9-e0563c63c371 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:57.601640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1429/2864] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:57.607608 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bdbbb-1296-4663-800b-f55f47cce547 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.625265 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bdbbb-1296-4663-800b-f55f47cce547 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:57.632778 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bee6c-bb3a-4c87-a977-49ca35fb7427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:57.634164 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bee6c-bb3a-4c87-a977-49ca35fb7427 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.634449 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bee6c-bb3a-4c87-a977-49ca35fb7427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.634716 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bee6c-bb3a-4c87-a977-49ca35fb7427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.635477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bee6c-bb3a-4c87-a977-49ca35fb7427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:57.637435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bee6c-bb3a-4c87-a977-49ca35fb7427 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:57.651517 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1436/2865] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:57.655895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1a7fd-5487-47d3-a7a9-e0563c63c371 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.666555 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad1a7fd-5487-47d3-a7a9-e0563c63c371 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:57.672933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9034c-165c-46dc-b283-c0d1d2e5d3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:57.673882 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9034c-165c-46dc-b283-c0d1d2e5d3b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.674165 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9034c-165c-46dc-b283-c0d1d2e5d3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.674444 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c9034c-165c-46dc-b283-c0d1d2e5d3b6 admin admin] PATH_INFO: `/v3/projects/6172a3c7510846208931ed29e9aa97e4/users/e427a768b5a4453c983a2e5f1ff6fcd5/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.696463 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9034c-165c-46dc-b283-c0d1d2e5d3b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6172a3c7510846208931ed29e9aa97e4, user_id=e427a768b5a4453c983a2e5f1ff6fcd5, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:57.701108 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c9034c-165c-46dc-b283-c0d1d2e5d3b6 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:57.719537 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1430/2866] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:57 2022] PUT /identity/v3/projects/6172a3c7510846208931ed29e9aa97e4/users/e427a768b5a4453c983a2e5f1ff6fcd5/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:57.731438 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c67f1c-924b-4f4c-b48a-8a30bffadae4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.731839 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c67f1c-924b-4f4c-b48a-8a30bffadae4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.732247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c67f1c-924b-4f4c-b48a-8a30bffadae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.759978 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-86c67f1c-924b-4f4c-b48a-8a30bffadae4 None None] MFA Rules not processed for user `e427a768b5a4453c983a2e5f1ff6fcd5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:57.761206 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-86c67f1c-924b-4f4c-b48a-8a30bffadae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:57.832070 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1437/2867] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:57.836838 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9034c-165c-46dc-b283-c0d1d2e5d3b6 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.909859 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c9034c-165c-46dc-b283-c0d1d2e5d3b6 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:57.915832 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f42bd-a9db-4e89-825e-220178533a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:57.916525 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f42bd-a9db-4e89-825e-220178533a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:57.916807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f42bd-a9db-4e89-825e-220178533a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:57.917080 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f42bd-a9db-4e89-825e-220178533a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:57.919082 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f42bd-a9db-4e89-825e-220178533a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:57.921342 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f42bd-a9db-4e89-825e-220178533a1e admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:57.951024 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1431/2868] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:35:57 2022] POST /identity/v3/projects => generated 401 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:57.955153 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c67f1c-924b-4f4c-b48a-8a30bffadae4 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:57.966930 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c67f1c-924b-4f4c-b48a-8a30bffadae4 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:58.007243 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:58.047504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778428ef-3660-48cb-916a-daade6f2700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:58.047504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778428ef-3660-48cb-916a-daade6f2700d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:58.047504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778428ef-3660-48cb-916a-daade6f2700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:58.047504 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778428ef-3660-48cb-916a-daade6f2700d admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:58.048095 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778428ef-3660-48cb-916a-daade6f2700d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:58.049645 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778428ef-3660-48cb-916a-daade6f2700d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:58.070897 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1438/2869] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:35:57 2022] POST /identity/v3/users => generated 307 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:35:58.077369 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f42bd-a9db-4e89-825e-220178533a1e admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:58.092216 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f42bd-a9db-4e89-825e-220178533a1e admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:58.097974 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570464a7-3886-4973-adc2-07d381ec9536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:58.098630 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570464a7-3886-4973-adc2-07d381ec9536 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:58.098916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570464a7-3886-4973-adc2-07d381ec9536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:58.099233 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570464a7-3886-4973-adc2-07d381ec9536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:58.099968 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570464a7-3886-4973-adc2-07d381ec9536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:58.101619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570464a7-3886-4973-adc2-07d381ec9536 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:58.113602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1432/2870] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:58.117955 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778428ef-3660-48cb-916a-daade6f2700d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:58.128811 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778428ef-3660-48cb-916a-daade6f2700d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:58.134637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bae6e-1b79-43a8-af20-835201935495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:58.135711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bae6e-1b79-43a8-af20-835201935495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:58.135874 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bae6e-1b79-43a8-af20-835201935495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:58.136188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913bae6e-1b79-43a8-af20-835201935495 admin admin] PATH_INFO: `/v3/projects/6fa8a72b578f4242b598d23b543ff94a/users/7cde842bdd5a44ba93ebc0f72189bb92/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:58.157269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bae6e-1b79-43a8-af20-835201935495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa8a72b578f4242b598d23b543ff94a, user_id=7cde842bdd5a44ba93ebc0f72189bb92, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:58.160047 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913bae6e-1b79-43a8-af20-835201935495 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:58.178971 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1439/2871] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:58 2022] PUT /identity/v3/projects/6fa8a72b578f4242b598d23b543ff94a/users/7cde842bdd5a44ba93ebc0f72189bb92/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:58.185913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570464a7-3886-4973-adc2-07d381ec9536 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:58.197142 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570464a7-3886-4973-adc2-07d381ec9536 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:58.264893 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84933b7e-9be5-4c9b-8172-159ce80a1f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:58.265640 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933b7e-9be5-4c9b-8172-159ce80a1f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:58.265950 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933b7e-9be5-4c9b-8172-159ce80a1f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:58.266269 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933b7e-9be5-4c9b-8172-159ce80a1f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:58.267059 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933b7e-9be5-4c9b-8172-159ce80a1f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:58.268910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933b7e-9be5-4c9b-8172-159ce80a1f54 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:58.279790 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1433/2872] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:35:58.284231 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bae6e-1b79-43a8-af20-835201935495 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:58.295236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913bae6e-1b79-43a8-af20-835201935495 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:58.300375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:58.301312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:58.301595 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:58.301862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] PATH_INFO: `/v3/projects/6fa8a72b578f4242b598d23b543ff94a/users/7cde842bdd5a44ba93ebc0f72189bb92/roles/b68ef721a7d6428e8b3fd70743791b8d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:58.303907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa8a72b578f4242b598d23b543ff94a, user_id=7cde842bdd5a44ba93ebc0f72189bb92, role_id=b68ef721a7d6428e8b3fd70743791b8d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:58.306695 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:58.318363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1440/2873] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:35:58 2022] PUT /identity/v3/projects/6fa8a72b578f4242b598d23b543ff94a/users/7cde842bdd5a44ba93ebc0f72189bb92/roles/b68ef721a7d6428e8b3fd70743791b8d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:35:58.332475 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8780-6841-4db5-bb9c-9b24695a4b51 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:58.332969 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8780-6841-4db5-bb9c-9b24695a4b51 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:58.333511 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8780-6841-4db5-bb9c-9b24695a4b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:58.345160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:35:58.370534 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d65a8780-6841-4db5-bb9c-9b24695a4b51 None None] MFA Rules not processed for user `7cde842bdd5a44ba93ebc0f72189bb92`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:35:58.372042 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d65a8780-6841-4db5-bb9c-9b24695a4b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:58.464827 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1434/2874] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:35:58 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:35:58.476264 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:58.478395 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:35:58.556634 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e095ebda-de13-4a2d-b343-0f22d484756e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:58.564823 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad3cc-6e3c-4962-88fc-2870c1a385ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:58.565997 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad3cc-6e3c-4962-88fc-2870c1a385ff service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:58.566511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad3cc-6e3c-4962-88fc-2870c1a385ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:58.566941 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ad3cc-6e3c-4962-88fc-2870c1a385ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:58.584468 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ad3cc-6e3c-4962-88fc-2870c1a385ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:58.587294 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ad3cc-6e3c-4962-88fc-2870c1a385ff service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:58.607107 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1441/2875] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:58 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:35:58.699602 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8780-6841-4db5-bb9c-9b24695a4b51 None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:35:58.717890 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8780-6841-4db5-bb9c-9b24695a4b51 None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:35:58.723594 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe14ab-6b0b-4802-a4f6-8a44bc382dd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:35:58.724579 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe14ab-6b0b-4802-a4f6-8a44bc382dd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:35:58.725122 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe14ab-6b0b-4802-a4f6-8a44bc382dd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:35:58.725647 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe14ab-6b0b-4802-a4f6-8a44bc382dd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:35:58.798125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe14ab-6b0b-4802-a4f6-8a44bc382dd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:35:58.799613 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe14ab-6b0b-4802-a4f6-8a44bc382dd8 service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:35:58.809258 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1435/2876] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:35:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:02.992613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad3cc-6e3c-4962-88fc-2870c1a385ff service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:03.070136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ad3cc-6e3c-4962-88fc-2870c1a385ff service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:03.075247 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:03.076152 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:03.076436 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:03.076721 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] PATH_INFO: `/v3/users/e427a768b5a4453c983a2e5f1ff6fcd5` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:03.078035 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e427a768b5a4453c983a2e5f1ff6fcd5)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:03.079964 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:03.117808 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:03.138552 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e427a768b5a4453c983a2e5f1ff6fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.139285 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e427a768b5a4453c983a2e5f1ff6fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.139892 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e427a768b5a4453c983a2e5f1ff6fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.152799 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e427a768b5a4453c983a2e5f1ff6fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.163204 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e427a768b5a4453c983a2e5f1ff6fcd5'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.176931 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1442/2877] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:02 2022] DELETE /identity/v3/users/e427a768b5a4453c983a2e5f1ff6fcd5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:03.181732 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615200df-f659-4160-abbf-a9d9c01f956b None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:03.182246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615200df-f659-4160-abbf-a9d9c01f956b None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:03.182246 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615200df-f659-4160-abbf-a9d9c01f956b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:03.200775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-615200df-f659-4160-abbf-a9d9c01f956b None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:03.202019 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-615200df-f659-4160-abbf-a9d9c01f956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:03.281578 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1436/2878] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:03.293097 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:03.294942 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:03.363168 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b4b2e-8e62-40e6-9c63-34c0f965f246 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:03.367239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3314e96-6713-4344-a1ab-f4153f3f197e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:03.367907 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3314e96-6713-4344-a1ab-f4153f3f197e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:03.368188 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3314e96-6713-4344-a1ab-f4153f3f197e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:03.368473 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3314e96-6713-4344-a1ab-f4153f3f197e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:03.379276 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3314e96-6713-4344-a1ab-f4153f3f197e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:03.381236 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3314e96-6713-4344-a1ab-f4153f3f197e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:03.395877 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1443/2879] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 17:36:03.693405 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615200df-f659-4160-abbf-a9d9c01f956b None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:03.695132 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-615200df-f659-4160-abbf-a9d9c01f956b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:03.711699 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615200df-f659-4160-abbf-a9d9c01f956b None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:03.717134 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:03.718070 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:03.718580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:03.718731 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] PATH_INFO: `/v3/projects/6172a3c7510846208931ed29e9aa97e4` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:03.719872 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6172a3c7510846208931ed29e9aa97e4)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:03.721853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:03.759775 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:03.794549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6172a3c7510846208931ed29e9aa97e4', 'request_id': 'req-f1eccde9-3150-4202-aa35-70e35ae9f91f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.803836 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6172a3c7510846208931ed29e9aa97e4', 'request_id': 'req-f1eccde9-3150-4202-aa35-70e35ae9f91f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.811427 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6172a3c7510846208931ed29e9aa97e4', 'request_id': 'req-f1eccde9-3150-4202-aa35-70e35ae9f91f'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.822813 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] The token cache is being invalidate because project 6172a3c7510846208931ed29e9aa97e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:03.823110 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.823785 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.826071 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1437/2880] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:03 2022] DELETE /identity/v3/projects/6172a3c7510846208931ed29e9aa97e4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:36:03.830190 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3314e96-6713-4344-a1ab-f4153f3f197e service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:03.840433 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3314e96-6713-4344-a1ab-f4153f3f197e service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:03.906367 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:03.907149 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:03.907426 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:03.907700 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] PATH_INFO: `/v3/users/7cde842bdd5a44ba93ebc0f72189bb92` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:03.909213 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cde842bdd5a44ba93ebc0f72189bb92)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:03.911279 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:03.970312 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cde842bdd5a44ba93ebc0f72189bb92'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.971014 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cde842bdd5a44ba93ebc0f72189bb92'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.971569 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cde842bdd5a44ba93ebc0f72189bb92'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:03.972728 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:03.992482 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cde842bdd5a44ba93ebc0f72189bb92'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:04.002048 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-28f4489c-6741-4cad-96b2-d151ccb3c6e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cde842bdd5a44ba93ebc0f72189bb92'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:04.016748 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1444/2881] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:36:03 2022] DELETE /identity/v3/users/7cde842bdd5a44ba93ebc0f72189bb92 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:04.353306 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:04.354964 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:04.370446 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eccde9-3150-4202-aa35-70e35ae9f91f admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:04.440849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:04.441808 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.442097 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.442387 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] PATH_INFO: `/v3/projects/6fa8a72b578f4242b598d23b543ff94a` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.443612 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa8a72b578f4242b598d23b543ff94a)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:04.445657 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:04.497598 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa8a72b578f4242b598d23b543ff94a', 'request_id': 'req-ae9842b7-6a82-40e7-8376-0faded6378b8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:04.507160 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa8a72b578f4242b598d23b543ff94a', 'request_id': 'req-ae9842b7-6a82-40e7-8376-0faded6378b8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:04.514777 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa8a72b578f4242b598d23b543ff94a', 'request_id': 'req-ae9842b7-6a82-40e7-8376-0faded6378b8'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:04.527056 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] The token cache is being invalidate because project 6fa8a72b578f4242b598d23b543ff94a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:36:04.527395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:04.528275 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-ae9842b7-6a82-40e7-8376-0faded6378b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:36:04.529910 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1438/2882] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:36:04 2022] DELETE /identity/v3/projects/6fa8a72b578f4242b598d23b543ff94a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:04.535323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0991-46fb-4930-b84d-7366478f1661 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.535613 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0991-46fb-4930-b84d-7366478f1661 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.535896 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6c0991-46fb-4930-b84d-7366478f1661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.559364 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fd6c0991-46fb-4930-b84d-7366478f1661 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:04.560615 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd6c0991-46fb-4930-b84d-7366478f1661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:04.624218 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1445/2883] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:04.632549 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9efe3-76e7-4b7c-bed2-927f8e03d4af None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.632795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9efe3-76e7-4b7c-bed2-927f8e03d4af None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.633098 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9efe3-76e7-4b7c-bed2-927f8e03d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.652688 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-19b9efe3-76e7-4b7c-bed2-927f8e03d4af None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:04.653955 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-19b9efe3-76e7-4b7c-bed2-927f8e03d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:04.662704 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1439/2884] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:04.667239 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c0991-46fb-4930-b84d-7366478f1661 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:04.677360 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6c0991-46fb-4930-b84d-7366478f1661 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:04.682203 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:04.682871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.683157 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.683627 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.684393 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 17:36:04.684711 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:04.686362 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:04.700020 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1446/2885] 10.222.0.66 () {58 vars in 1152 bytes} [Thu Jun 16 17:36:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 17:36:04.704607 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9efe3-76e7-4b7c-bed2-927f8e03d4af None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:04.714958 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9efe3-76e7-4b7c-bed2-927f8e03d4af None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:04.720374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1b5a-651d-49bb-a5fa-a1f7fe89f5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:04.721037 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1b5a-651d-49bb-a5fa-a1f7fe89f5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.721353 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1b5a-651d-49bb-a5fa-a1f7fe89f5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.721619 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd1b5a-651d-49bb-a5fa-a1f7fe89f5d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.722519 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd1b5a-651d-49bb-a5fa-a1f7fe89f5d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:04.725141 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd1b5a-651d-49bb-a5fa-a1f7fe89f5d5 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:04.753849 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1440/2886] 10.222.0.66 () {60 vars in 1152 bytes} [Thu Jun 16 17:36:04 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:04.758687 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:04.770363 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d628cc8-1232-4e47-883e-99ee68b8e6b8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:04.834450 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378828c-f320-4f46-9eb3-fca65022e613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:04.835132 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378828c-f320-4f46-9eb3-fca65022e613 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.835411 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378828c-f320-4f46-9eb3-fca65022e613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.835684 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378828c-f320-4f46-9eb3-fca65022e613 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.836355 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378828c-f320-4f46-9eb3-fca65022e613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:04.838315 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378828c-f320-4f46-9eb3-fca65022e613 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:04.870160 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1447/2887] 10.222.0.66 () {60 vars in 1143 bytes} [Thu Jun 16 17:36:04 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 17:36:04.875346 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1b5a-651d-49bb-a5fa-a1f7fe89f5d5 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:04.886625 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd1b5a-651d-49bb-a5fa-a1f7fe89f5d5 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:04.891807 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec3f63-a419-4002-a924-c4ca3eada096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:04.892488 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec3f63-a419-4002-a924-c4ca3eada096 admin admin] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.892774 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec3f63-a419-4002-a924-c4ca3eada096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.893050 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec3f63-a419-4002-a924-c4ca3eada096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.893804 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec3f63-a419-4002-a924-c4ca3eada096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:04.895411 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec3f63-a419-4002-a924-c4ca3eada096 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:04.908436 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1441/2888] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:04.913398 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378828c-f320-4f46-9eb3-fca65022e613 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:04.924697 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378828c-f320-4f46-9eb3-fca65022e613 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:04.930590 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418743dd-eca7-4fec-81b4-30c661878f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:04.931269 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418743dd-eca7-4fec-81b4-30c661878f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.931554 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418743dd-eca7-4fec-81b4-30c661878f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.931879 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418743dd-eca7-4fec-81b4-30c661878f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.932630 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418743dd-eca7-4fec-81b4-30c661878f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:04.934307 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418743dd-eca7-4fec-81b4-30c661878f7d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:04.945637 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1448/2889] 10.222.0.66 () {58 vars in 1121 bytes} [Thu Jun 16 17:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 17:36:04.950847 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec3f63-a419-4002-a924-c4ca3eada096 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:04.957301 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:04.965685 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec3f63-a419-4002-a924-c4ca3eada096 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:04.971956 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:04.972905 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] REQUEST_METHOD: `PUT` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:04.973222 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:04.973505 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] PATH_INFO: `/v3/projects/09d39fa8db0747f9b8237807b2735d5c/users/6833efa3f23d44a6b10d6b82e0b2b38c/roles/27d4ad049d8f4ff4a176581dea972512` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:04.995447 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d39fa8db0747f9b8237807b2735d5c, user_id=6833efa3f23d44a6b10d6b82e0b2b38c, role_id=27d4ad049d8f4ff4a176581dea972512)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:04.998431 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:05.016119 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1442/2890] 10.222.0.66 () {60 vars in 1482 bytes} [Thu Jun 16 17:36:04 2022] PUT /identity/v3/projects/09d39fa8db0747f9b8237807b2735d5c/users/6833efa3f23d44a6b10d6b82e0b2b38c/roles/27d4ad049d8f4ff4a176581dea972512 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 17:36:05.022477 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06b666-0c28-4d89-9fae-935a50025229 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:05.023345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06b666-0c28-4d89-9fae-935a50025229 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:05.023345 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06b666-0c28-4d89-9fae-935a50025229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:05.043375 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1e06b666-0c28-4d89-9fae-935a50025229 None None] MFA Rules not processed for user `918e7fd9b64348e38ccb56232b8e098a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:05.044779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1e06b666-0c28-4d89-9fae-935a50025229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:05.073074 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:05.122801 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1449/2891] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:05.133016 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:05.134965 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:05.195698 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd0d22-4a0e-46f3-b433-9af4eb1f8402 admin admin] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:05.199925 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72fb6996e1974060b162070f8da365aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:05.200592 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:05.200934 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:05.201221 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:05.213350 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:05.215192 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:05.230207 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1443/2892] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:06.008182 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f771d-08d4-4e5d-99df-68c294a13676 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:06.008464 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f771d-08d4-4e5d-99df-68c294a13676 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:06.008575 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f771d-08d4-4e5d-99df-68c294a13676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:06.043934 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-091f771d-08d4-4e5d-99df-68c294a13676 None None] MFA Rules not processed for user `6833efa3f23d44a6b10d6b82e0b2b38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:06.045217 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-091f771d-08d4-4e5d-99df-68c294a13676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:06.137495 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1450/2893] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:36:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:06.149842 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:06.151347 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:06.215900 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2c526-3ac8-48a8-9ac3-d2dcec364347 service neutron] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:06.219913 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6179dc2bd1d4ff2a04a8362d63e7b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:06.220562 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:06.220844 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:06.221120 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:06.231186 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:06.232638 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:06.242812 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1444/2894] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:07.642904 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63d2e-4891-41f1-a4f3-280563b7753b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:07.643836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63d2e-4891-41f1-a4f3-280563b7753b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:07.643836 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e63d2e-4891-41f1-a4f3-280563b7753b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:07.664061 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c3e63d2e-4891-41f1-a4f3-280563b7753b None None] MFA Rules not processed for user `6833efa3f23d44a6b10d6b82e0b2b38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:07.665335 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c3e63d2e-4891-41f1-a4f3-280563b7753b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:07.673883 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1451/2895] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:36:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:08.872276 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:08.874033 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:08.983125 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10097db-84aa-4be3-9a64-8c8565a40e7a service cinder] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:08.988895 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229673b9-40f2-464b-96a4-0746253f45b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:08.990075 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229673b9-40f2-464b-96a4-0746253f45b2 service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:08.990355 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229673b9-40f2-464b-96a4-0746253f45b2 service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:08.990631 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229673b9-40f2-464b-96a4-0746253f45b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:08.992603 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-229673b9-40f2-464b-96a4-0746253f45b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:09.001923 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:09.014632 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229673b9-40f2-464b-96a4-0746253f45b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:09.016281 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229673b9-40f2-464b-96a4-0746253f45b2 service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:09.027761 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1445/2896] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:36:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:36:17.657947 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=48 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:19.022567 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=3 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:24.328919 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2161e1f-ef49-47c6-9c4f-22057c096d22 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:24.329514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2161e1f-ef49-47c6-9c4f-22057c096d22 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:24.329514 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2161e1f-ef49-47c6-9c4f-22057c096d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:24.348805 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c2161e1f-ef49-47c6-9c4f-22057c096d22 None None] MFA Rules not processed for user `6833efa3f23d44a6b10d6b82e0b2b38c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:24.350038 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c2161e1f-ef49-47c6-9c4f-22057c096d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:24.358604 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1452/2897] 10.222.0.66 () {58 vars in 957 bytes} [Thu Jun 16 17:36:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:34.343472 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:44.755183 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fe237-7d3e-446c-a12c-03b2a1b35fb4 None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:44.755689 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fe237-7d3e-446c-a12c-03b2a1b35fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:44.755980 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716fe237-7d3e-446c-a12c-03b2a1b35fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:44.834978 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-716fe237-7d3e-446c-a12c-03b2a1b35fb4 None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:36:44.836580 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-716fe237-7d3e-446c-a12c-03b2a1b35fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:44.915069 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1446/2898] 10.222.0.66 () {58 vars in 1009 bytes} [Thu Jun 16 17:36:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:36:44.979263 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2161e1f-ef49-47c6-9c4f-22057c096d22 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:36:44.980994 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c2161e1f-ef49-47c6-9c4f-22057c096d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:36:45.065240 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2161e1f-ef49-47c6-9c4f-22057c096d22 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:36:45.070858 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc9764-07d8-4f2d-9b86-3ec5431abff8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:36:45.071499 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc9764-07d8-4f2d-9b86-3ec5431abff8 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:36:45.071779 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc9764-07d8-4f2d-9b86-3ec5431abff8 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:36:45.072050 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc9764-07d8-4f2d-9b86-3ec5431abff8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:36:45.081016 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc9764-07d8-4f2d-9b86-3ec5431abff8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:36:45.082618 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc9764-07d8-4f2d-9b86-3ec5431abff8 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:36:45.091876 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1453/2899] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:36:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:36:54.911776 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=47 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:36:55.087612 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=36 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:00.352434 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225170d-0c64-42fa-b017-253e09a5d57e None None] REQUEST_METHOD: `POST` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:00.352916 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225170d-0c64-42fa-b017-253e09a5d57e None None] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:00.353022 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225170d-0c64-42fa-b017-253e09a5d57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:00.371394 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.auth.core [None req-d225170d-0c64-42fa-b017-253e09a5d57e None None] MFA Rules not processed for user `ae4e4bbf4359400289b9fd2ba8778067`. Rule list: `[]` (Enabled: `True`). {{(pid=80239) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 17:37:00.372600 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-d225170d-0c64-42fa-b017-253e09a5d57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:00.380938 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1447/2900] 10.222.0.66 () {58 vars in 967 bytes} [Thu Jun 16 17:37:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 17:37:00.394023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc9764-07d8-4f2d-9b86-3ec5431abff8 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:00.407435 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc9764-07d8-4f2d-9b86-3ec5431abff8 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:00.412551 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53aa2f-290c-4b95-bac8-c93f5e430bba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f15957f640a411eb4ac8861454f2b97', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:00.413223 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53aa2f-290c-4b95-bac8-c93f5e430bba service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:00.413510 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53aa2f-290c-4b95-bac8-c93f5e430bba service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:00.413789 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53aa2f-290c-4b95-bac8-c93f5e430bba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:00.422511 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53aa2f-290c-4b95-bac8-c93f5e430bba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:00.424099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53aa2f-290c-4b95-bac8-c93f5e430bba service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:00.433724 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1454/2901] 10.222.0.66 () {60 vars in 1384 bytes} [Thu Jun 16 17:37:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 17:37:02.351219 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225170d-0c64-42fa-b017-253e09a5d57e None None] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:02.364933 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225170d-0c64-42fa-b017-253e09a5d57e None None] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:02.370483 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:02.371109 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] REQUEST_METHOD: `GET` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:02.371395 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:02.371669 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:02.374268 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.fernet_utils [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80239) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:37:02.387247 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:02.388885 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:02.399901 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1448/2902] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:37:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:37:02.418023 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53aa2f-290c-4b95-bac8-c93f5e430bba service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:02.430585 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53aa2f-290c-4b95-bac8-c93f5e430bba service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:02.440391 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:02.441328 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:02.441610 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:02.441871 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] PATH_INFO: `/v3/users/6833efa3f23d44a6b10d6b82e0b2b38c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:02.443112 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6833efa3f23d44a6b10d6b82e0b2b38c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:02.445018 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:02.499447 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6833efa3f23d44a6b10d6b82e0b2b38c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.500284 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6833efa3f23d44a6b10d6b82e0b2b38c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.501003 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6833efa3f23d44a6b10d6b82e0b2b38c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.518136 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6833efa3f23d44a6b10d6b82e0b2b38c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.527340 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6833efa3f23d44a6b10d6b82e0b2b38c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.540981 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1455/2903] 10.222.0.66 () {58 vars in 1223 bytes} [Thu Jun 16 17:37:02 2022] DELETE /identity/v3/users/6833efa3f23d44a6b10d6b82e0b2b38c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:37:02.724392 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] Authenticating user token {{(pid=80239) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:37:02.739539 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed2686-84b9-4956-8c33-3d26d5aa1081 service placement] Validating token access rules against request {{(pid=80239) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:37:02.804833 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '918e7fd9b64348e38ccb56232b8e098a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '546ad1026cd44cbea571239c12f41a3f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80239) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:37:02.805735 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:37:02.806023 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:37:02.806304 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] PATH_INFO: `/v3/projects/09d39fa8db0747f9b8237807b2735d5c` {{(pid=80239) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:37:02.808150 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d39fa8db0747f9b8237807b2735d5c)` {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:37:02.810094 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] RBAC: Authorization granted {{(pid=80239) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:37:02.859374 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d39fa8db0747f9b8237807b2735d5c', 'request_id': 'req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.867853 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d39fa8db0747f9b8237807b2735d5c', 'request_id': 'req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.874795 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d39fa8db0747f9b8237807b2735d5c', 'request_id': 'req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd'} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.886255 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] The token cache is being invalidate because project 09d39fa8db0747f9b8237807b2735d5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80239) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 17:37:02.886541 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.887334 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG keystone.notifications [None req-70ad5744-2be0-4738-99e1-fc0d1ed5bdfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80239) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 17:37:02.888967 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: [pid: 80239|app: 0|req: 1449/2904] 10.222.0.66 () {58 vars in 1232 bytes} [Thu Jun 16 17:37:02 2022] DELETE /identity/v3/projects/09d39fa8db0747f9b8237807b2735d5c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 17:37:12.537323 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:37:12.882939 dv-cinder-837356-2-iscsi devstack@keystone.service[80239]: DEBUG dbcounter [-] [80239] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80239) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 17:41:43.469212 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 17:41:43.470870 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:43.547523 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76dc46d-08ed-4916-a505-f7c3096b99c2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 17:41:43.611301 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a43f21-4470-496a-8575-6dc89b79fa6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7a2b40b05a6846099bd4c66609365ad5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8deb4e225ab24d98bacff11a9f183d8c', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 17:41:43.612099 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a43f21-4470-496a-8575-6dc89b79fa6c service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 17:41:43.612572 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a43f21-4470-496a-8575-6dc89b79fa6c service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 17:41:43.613094 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a43f21-4470-496a-8575-6dc89b79fa6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 17:41:43.615341 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c9a43f21-4470-496a-8575-6dc89b79fa6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 17:41:43.627549 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a43f21-4470-496a-8575-6dc89b79fa6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 17:41:43.629408 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a43f21-4470-496a-8575-6dc89b79fa6c service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 17:41:43.639862 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1456/2905] 10.222.0.66 () {60 vars in 1365 bytes} [Thu Jun 16 17:41:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 17:41:53.635562 dv-cinder-837356-2-iscsi devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=58 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}